isn't pretty much all speciation a result of a distinct sub species breeding in isolation for so long from another that they reach the point where they are no longer fertile with the the other subspecies?

like other than the weird mutational loading and stuff is there any real downside to generations upon generations of inbreeding?