"Things got so bad that when I went shopping for a house, some people would refuse to open the door if they saw it was me standing there. And drunks would always want to challenge me."