I have say it is pretty bad. I grew up in Hong Kong with clear blue skies (as a kid) and then came back some 20 odd years later to a much different environment. Being away for so long I really noticed the difference. Summer is better than winter due to the change in trade-winds. This past winter was pretty awful...my work takes me outdoors most days and there were a couple of days where I got physically ill. Now, it's gettig better and past week has improved. You will see blue skies more often in the summer-time.
There's alot of things about HK that are fantastic (nightlife, business, socializing, shopping etc...), but air quality is not one of them.