According to the DDD retcons, as soon as Xehanort split himself in two and became Ansem:
-He went back in time as a potato-man without a physical body
-Gave his younger self time travel powers.
-Spent like 80 years sitting around in Destiny Islands as a weird potato-man.
-Talked to Sora in the cave.
-Put the X in time-travelling Sora from DDD.
-Possessed Riku at some point.
So he never really went back to his own time, he just waited it out for 80+ years until Sora killed him.