to answer your question:
There's no way in hell that, at the age of 4, you had enough context to watch a mouse with no human features below the neck and derive arousal when it stops wearing a frock.
If there's any truth to this scenario, it's that you either had one of those random kid-boners and ascribed cause-and-effect to it, OR you had an older brother who reacted to it, and you took cues from him.
now, Minnie Mouse's bloomers, on the other hand -- she is also shapeless, but little boys are told not to look at girls' underwear on purpose. so when I was like 6 or 7 I was excited to see them because I was getting away with something. but I don't remember getting an actual boner that way.