Aku rode on his invulnerability to pretty much anything but the sword. Sure the elemental spirits flexed on him in combat, but as it has already been said, they weren't able to truly wound him either (he just got tired of getting pummeled)
It was ultimately a matchup thing. Aku was pushed around by the spirits, who were defeated by Jack, who was shat on by the Guardian, who was defeated by Aku. The setting was fucked outside of Jack because nobody had a sacred artiifact on the level of the sword for some reason, so Aku could safely spam shit like his rain of stakes and destroy most things, and what little he couldn't destroy himself he easily avoided and continued on, since he could jump between planets in the blink of an eye.