How do we know chloroplasts are newer than mitochondria? What if the ancestor to fungi and animals just lost the chloroplasts at some point?

Maybe this is a better place than /an/ to ask this question?