I have been separated from my husband for 1 year we have a 3 year old daughter who has lived with me only?

and he has had access to her only a weekly basis for the whole day only as i felt she was a toddler and dependant on me. He wants to have her nights and has said he can take her when he want and that he can by law have half custody of her. She goes nursery 4 days a week so for her best interest its best he sees her weekends however i want things done properly and said to him he can only have her nights when we have sorted things via mediation or court. He is controlling, critising me and brining me down in general, cold basically i was in a loveless marriage with someone who didn't care about me and because he would not leave the house and he kept saying to me if i am not happy you know where the door is, so that's what i did. I don't trust him so need advise. He has said if he cant take her this weekend he is going to order a court order. I told him do what he must but I know he is going to play nasty, not that i have anything to be scared of but he is making out i am agressive and god knows what else. He is very controlled and before he was happy because i didn't take him for the money the house etc but now its about my daughter so i must fight even though its only a trophy thing with him.