Tadano confessed to the hottest girl in class back in middle school.
Tadano still has confidence issues about romance due to that.
Remember, Tadano's true self is a Gangster Chuuni Chad. He suppressed his true personality once he entered highschool into the most plain, average guy as much as possble.
Had that middle school rejection never happened, Tadano would be more spunky and hyperactive and his hair and clothes would be much more aggressive looking.