>>117843812If not on the day of the comet, Bumi stomps him. Hell, Aang could have stomped him if not for the comet, and it's treated almost like common knowledge Iroh could have kicked his ass if he'd ever really wanted to, at least in his prime.
The fear at the climax wasn't that Ozai couldn't be stopped but that he'd kill millions before he could be.