It's not about height itself.
Put yourself in his place.
You are the best paid actor in Hollywood in one of the most popular franchises right now. You could buy the fastest car, the biggest house, bang the hottest chick, whatever.
BUT, the only thing you could never get, is being tall.
No matter what you do, no matter your hairstyle or your shoes, you'll always be a 5'6 manlet.
You could get that surgery that makes you taller, but everyone knows it will fuck up your legs permanently and your ability to walk will be hindered for life.
It's not only the fact of being smaller than the rest, is the inability to fixing that, even if you have that much power.