If you already have to raise kid then it is hard. It is hard but not "impossible". Not just for men, I witness many female have to take care of their newborn baby yet at the same time doing their phd when they were in the program, intensive research workload, TA/RA duty, admin duty...etc.
I don't know which subject you are undertaking. But one principle is universal to most phd, that is, if you want to earn a lot and be a big bug, go to industry; if you want to go back to academia after graduation and enjoy the reputation at school plus free working mode in university then do your phd. Otherwise you will be painful/discouraged when you see many peers earn a lot already. Phd is a very persistent and patient process especially you need to face the possibility of program extension to 5 years, the change of thesis topic, plus pressure of publication in some department.
My suggestion is, you can talk with your colleague, including your phd classmate, your own supervisor, program coordinator, etc., through orientation at the very first beginning of semester, or through research seminar/ workshop, to observe that whether that is the life style that you want to chase. Your concern is not unique, many ppl have the same consideration before they made their final decision.
Hope you figure out soon.