I'm someone else who's quite happy living in Tuen Mun district, although now at the opposite end of it from Lam Tei.
It's true that the beaches aren't a patch on eastern Australia or Hawaii, but they're no worse (and possibly better) than many places that go by the name "beach" in the UK. There are signs up saying that it's advisable not to swim for a few days after heavy rain, but IMHO these are past their use-by date as they are survivors from when there were piggeries in the surrounding hills. Water quality is touch and go: yesterday Golden/ Cafeteria beaches weren't too bad, but the water just around the headland at Kadoorie was full of stuff (mostly leaves and twigs, admittedly). Castle Peak was good.
What I will say is that this corner of Hong Kong is "quiet" by LOCAL standards; coming from rural England it will still seem pretty "hyper"