That's not how it worked though
The worms just made him more able to do the things he wanted to do.
He could play the music that was already in his head, put his feelings into words better than before but still originally his.
They didn't change his personality or thoughts, just improved his body and made his brain work better, but not differently.
He kicked them out not because he wasn't in control of himself or they were changing him, but because he wanted to know if Leela liked him for him or just because he was suddenly suave and romantic as fuck and Leela is a bit of a vapid whore who is easily swayed by shit like that.
It was >here's a million bucks>No I don't want it because then I'll never know if Leela loves me for me or the money
Still though, probably better to get rid of them because they'd end up having some final stage of their life cycle where you explode and scatter their spores everywhere or something.