This is definitely part of it, especially with the dominant /ss/. He can be manly without being a man, as it were. Part of that is just that actual men remind you of the trappings of actual adulthood, which you probably suck at.
But also, being actually powerful is manly, but real power is kind of morally suspect. If you have just a regular grown-ass hairy man being dominant, it might remind you of your insecurities, but it might also remind you of being given stern words about oppression or wife-beating a bunch of times. You've been told that woman want some kind of revenge on powerful men, or that it will frighten them and make them sad.
But if it's just some kid, it's fine. For one thing, any manliness at all is hyper-masculine because it's infinity percent of what's expected. And, he can be as smug as fuck without it seeming un-fun.