Abusive step son

Hi, Hubby and I have been together for 8 years. He has a now, 13 year old son, he came to live with us when he was 7 for 4 years and went back to his mums 2 years ago as he didnt want to live with us anymore. She was pushing for the move.. He came to us cause his mum wasn't coping, she told us to have him to see what it was like. She had no rules in place, he was constantly in trouble at school, watched what he wanted, did what he wanted etc.. He is a lot of hard work, but after 2 years he settled down, started passing school, being respectful all the things we hoped for.. We where strict and always have been, he was quite proud of his achievements.. Anyway while he was living with us we also had 2 kids who are now 4 and 3!! Its been crazy!! He was jealous which is to be expected.. Also while he lived with us he went to a physc as there was a lot going on for him and we thought it would be good for him to off load to someone not involved in our lives..During this time we where told to never leave our children unattended with him, from both the physc and his school pshyc because of some behaviours he exhibited at school and home.. My husband also works away from home was was only home for one week out of five for 2 years.. Anyway when he went back to live with his mum he changed, the day he left our home it was bad.. He completely lost respect for us and any rules we had in place at home.. Only 6 months ago we where lead to believe that he sexually abused our 2 kids and that it happened while he was visiting for the weekend..They said and done things that pointed to this and we got it checked out and although it wasnt extreme it still happened.. Since we've found out, I no longer have anything to do with him and neither do our kids.. Our kids are alot more relaxed, both kids are toilet trained day and night and bed wetting has stopped, more calmer etc.. It happened in our home while kids where outside, very sneaky and deceptive.. Now he has been asked and his mother told but all reactions have been, they are all just kids.. I believe given his age that he knows right from wrong.. His mother now wants us to have him back as shes not coping, but I don't want anything to do with him. My hubby still sees him and I would never stop him..But now my husband is almost blaming me for this and wants his son back...There is a lot more to this, but its 8 years of stuff, this is the short story of the biggest issue..Has anyone else been through this, what would you do? Im only asking for peoples thoughts on the topic and not judgement on our lives..Thanks