Advice needed!!!

Okay, I have a question for all and really need opinions, p.s so sorry for the long story. I currently share care my 2 daughters and have done for the last 5 years. The last year and a bit their father has seriously gone down hill. He refused to take them to their extra curricular activities giving me the excuse he couldn't afford it and didn't have the time. So I paid (or more my husband paid for them) my eldest daughter has been playing soccer for 4 years (and is amazing at it) so it wasn't a new suggestion she do it, and our other daughter had started gym the year before and was to continue this year. Anyway he came back telling me he wouldn't or couldn't take them on his weeks as 1. he couldn't afford it and 2. he didn't have the time. I therefore told him it had already been paid and offered to take them either have them at my house the extra nights or pick them up from school, take them to their activities and then drop them off but no matter my offers none of them were accepted. When I asked my girls what they did at those times they said nothing! He sits in his room while they sit out in the lounge room with no tv or activities mind you (words from their mouth not mine as I have no idea what happens over there). This has been going on for months now and last night they told me he and his house mate (Who is an older lady and my girls have informed me they share a room plus relayed a few sexual things they have seen them doing) smoke in their ensuit, I asked why and what the bathroom looked like and they said that it was filthy and drew me examples of their smoking utensils, in my opinion it was a bong, a long tubed implement with a spout and handle. I even messaged my long term friend who I speak to whenever we get the chance as we have 7 kids amongst us and she said they looked like that. No coaching on my part. My question now is what do I do? My girls have said they don't want to go their anymore as he eats food in front of them and just stays in the office or his room and rarely spends time with them. There are no custody or court orders. I tried mediation but he never showed up. I did get a certificate but have 2 young kids under 3 and found it hard to organise the whole getting to court process (they don't make it easy that is for sure). I would appreciate any advice and specific wording I should be saying to him, I always feel he out talks me and it does my head in. He is a manipulator and always has been (I even recall him saying when we were together I should be grateful he doesn't hit me like all my friends partners do!)