Custody

Ok long story short
I have a 4 yr old daughter
She has always lived with me
Her biological father has mental issues and drug dependencies as in ice, pot, and pretty much everything else
He's been in and out of jail
He is also aboriginal
He was extremely violent to me while we were together and when we had our daughter I left him
I have been to serveral legal aid services and lawyer advice places
They all say I cannot get full custody on paper cos he has attempted to get custody
I cannot take her on a holiday cos I cannot get a passport
He has full rights to take her and leave at a minutes notice
He is not a suitable or safe person to be around her
He has never had her by himself but he and his family have made threats to take her when I least expect it
Legal aid says unless he makes a court date to file I cannot
I think that's ridiculous
I am terrified for her at school as they have said cos he's on her birth certificate he can legally pick her up and disappear even though he isn't on the list of people who can
I'm terrified to walk through the street with her incase he turns up and snatches her
How do I fix this
He only wants her for spite and I know he cannot care for her
Help please