What unforgivable act do the radleys commit which makes the town suspicious of them?

After Boo, as a kid, committed an act of violence the Radleys just kind of shut him up in the house and basically kept him as a prisoner for the rest of his growing up. The town of Maycomb was as much curious about the family as they were suspicious.