Penguin and Riddler are on the same boat because they were the breakout stars of the Adam West show, and when it was decided that Batman had to be purged of camp all together, Eddie and Oswald effectively became the sacrificial lambs, cast aside to make way for the new badasses like Ra's al Ghul. Riddler was made into a complete joke and Penguin became a "rancid momma's boy and bully", by O'Neil's own words. They still had great stories afterwards, mostly because of Alan Grant and Chuck Dixon, but they still carry that stigma today.
In fact the reasons why Joker survived the reputation of the show was because he already had roots in being a murderous, horrifying crime boss mastermind in his first Golden Age appearences, and because he absorbed major aspects of Frank Gorshin's Riddler (who was far more like the Joker we think of today than the Riddler as we know him), making him better than ever and leaving Riddler with the scraps. Which is part of why people like >>106110871
tend to write off Riddler as a poor man's Joker.
Two-Face is a different case, since he wasn't on the show and you could argue he was both better off, since he got to remain a grotesque and tragic major villain through the 70s and 80s, or worse, because it wasn't until BTAS and especially (unfortunately) Forever that he became famous at large. And while I'd argue he was better off not being in the show, his relative lack of popularity amongst casuals might have contributed to other villains occupying his roles or stealing his gimmicks.
Two-Face used to be the major tragic Batman, a role now occupied by Mr.Freeze and many others. His deformity made him stand out, until characters like Killer Croc or Clayface arrived. He was the first villain to be "Bruce's childhood friend", until characters like Black Mask and Hush (ugh) came along, and so on. Plus, he's had too many bad or lackluster reinterpretations, which sadly misinform and sour fans about him.