So because it's an official language, you think the general population will/should speak it? There's absolutely no good reason for most locals to speak English so why should they? HK is now China, the majority are far better off learning mandarin than they are learning English.
French is an official language in Canada and I can tell you that that HKer speak far better English than most Canadian speak French.