if I cut through Kowloon city and follow the signs through to Kwai Chung / Lantau, it takes me 40 minutes on average, sometimes 35, but not longer than 45 mins. I do this trip quite regularly and don't mind it. We not so long ago decided to down size our car and be a little more economical, buying a Toyota Prius, and have to say, its cheaper than using public transport, and a lot more comfortable. Pretty happy with it.
The drive is actually pretty easy, once you learn the short cuts. I am happy to draw up a route map for you via Google Maps, as there are a couple that I use depending on what time of day it is, to avoid congestion on Hirams highway ( the main road that runs through Sai Kung from Marina Cove to the beginning of Tai Mong Tsai road)..