I need help please

so I feel I’m being gaslighted again so can someone help me figure how this is logical?

I was in a DV relationship and that DV cycle kept happening, the I’m sorry I love you it won’t happen again thing, then it’s good then it happens again.

So I move away 2 hours north after being hit yet again got an AVO but made it so his daughter was not on there, and he could come near me to access his daughter.
He has left before and she didn’t cope well. So it’s scared me into making sure she sees him.
I moved to where I have friends and family to break this cycle I had no support down there.

I found a house and transferred my job the kids are in school and little one in daycare.

I put on paper that our daughter goes to him every weekend even though he’s not there for one day of that weekend, I’ve stuck to it even though my baby is exhausted from all of the driving she needs to see her dad she really loves him to pieces and he has never hurt her only me and my older kids.

I’ve asked for 3 weekends in a month but he won’t budge.

He came to me with a proposal “I’ll pay your break lease your moving expenses and half your rent for a year if you move back to Sydney.” I said “well if you can afford all of that why can’t you get a house closer?”

My response was No!
Which was then met with damage to my garage door and me having him shut me between the door to the house and the Frame with my kids screaming getting worried he was hurting me.

I didn’t breach him on this because I have tried before and the police say there weren’t witnesses other than the kids we can make a note but can’t breach him.

I told him “I have kids settled and I’m not moving down” I said “you move up to make it easier on the baby.”

he lives with his parents in western Sydney and has a job in The shire.
his mum is sick but his sister is her carer through Centrelink she lives in the same suburb.
My question is who do you think should move to make it easier on our daughter?
He even got his best friend on the phone who actually agreed it should be me that moves because the baby was born down there!! huh??
Am I wrong or should he be the one to make it easier on our daughter or me?
I feel like I’m going insane I just want to go to mediation to end all of this crap but I know he won’t agree to 3 weekends a month And he lives there rent free so I can see that’s a factor in him pushing me to move. I really feel nuts after the fact he’s getting friends to tell me he is in the right?
Please help me?

Btw I am applying for the AVO to be changed that he has to meet me at McDonald’s for pick ups. Not near my house.