Why are we even talking about living in remote parts of the New Territories or whatever as a possibility for someone who is incoming to HK with a newborn baby?
You absolutely need a car to do this, otherwise your life becomes a misery of waiting for crowded and infrequent public buses. Doing this in the height of a humid summer with a stroller or (even harder) a baby carrier is absolutely miserable. Just writing this gives me flashbacks of trying to keep a baby cool outdoors during summer.
Even with a car, social circles will be far harder to establish as people will need to drive all over the place to meet, and there are simply less people out there (which is what makes it attractive). In a big estate, you can literally just go down an elevator and walk to a playroom to talk to other parents or for the kid to find others to play with (this won't be relevant until the kids is closer to 2).
Not having to own a car, imo, is one of the big draws of living in HK.
I know people with kids out in the New Territories. It works for them, but they had all lived in HK for years prior, all started out in more convenient places and built up networks, figured out how the city works. And socialising became much more limited once they moved out. One mother who moved out to Saikung made an interesting observation- she found other stay at home mothers to hang out with, but mainly did so for the benefit of her child. The social pool was smaller there, and she missed being on HK Island where she could find people that she had more than simple parenthood in common with.
From the sounds of it, the husband will be at work most of the week. The mother is going to be figuring out most of the incredibly confusing 'new baby' stuff and just 'popping out' in the car for a quick meet up will be a huge challenge- the baby will be feeding and falling asleep every couple hours, and the whole extended nursing/burping/changing routine is fairly exhausting. And if you're meeting other parents, they're subject to the same routines but possibly with different timings. Being trapped in the boonies without anyone to talk to most of the time sounds like a fast track to post natal depression.