Fuyumi probably quit because it's a huge stigma to be related to a criminal in Japan and she probably got pushback because of that and some parents might have not wanted her around their kids. Stuff like that is why Togas parents were quick to disown her publicly. The fact that the Todorokis stayed by Toya's side is a huge deal.