The 40,000 figure has plenty of room for all the Brits, Australians, Canadians, Americans and Indians claiming "English" as their "usual" language at home according to the 2006 Census (and the number of native English speaking will be lower than that, particularly for Indians). There really aren't very many expats in HK if you exclude the domestic helpers (almost none of whom have kids here)
There are only 11,000 residents of HK who are not one of Chinese, South Asian (Indian sub-continent), Thai, Filipino, Indonesian, Japanese, Brit/US/Canadian/Aussie.