>>3430673>>3430674=== SPOILER ALERT FOR THOSE WHO HAVEN'T READ/WATCHED IT ===
It's only implied in the manga that Tsukasa constantly abused Amane when they were alive, the sexual implications of the abuse came from certain scenes in both the manga and the anime mixed with the dirty minds of some /a/nons.
Still, there are some suggestive undertones all over the manga and anime that can back up the rape theory, like the way Hanako reacts to Tsukasa every time he touches him (literal PTSD), Tsukasa's way of interacting with Hanako (sometimes he's super clingy, and sometimes he's like the rapists in L&O SVU), that scene in the opening of the anime where Tsukasa has Hanako chained and moves his hand towards his crotch while biting his ear, etc.
Not to mention that things like the covers of the manga looking like they're from porn doujins (seriously, Hanako always has these "fuck me" eyes and hugs someone in the covers) and Hanako constantly pulling lewd pranks on Nene and Kou only fuel the imagination of perverted readers (like me :D) even more.