When do you know it's time to end a relationship...kids involved..?

I have been living with him for one and a half years and we have together a two year old daughter. I also have an 11 year old son from my first marriage. We were going to get marriage but I stopped everything since I am not sure if I'm doing the right thing. I think I he does not love me and is only with me because of our daughter...he has three boys from previous marriages and this is his first and only daughter. He does not get along with my son and diminish him whenever he gets a chance. He is not happy of my triumphs at any moment and only cares about him. He is a very selfish person who thinks that deserves the world. I want to end the relationship and move on with my life but I dont know if I will regret it. Please help.