Tom - your concerns are understandable, especially if you haven't moved before, but fear not. Frankly I would look at this from a different angle -- right now is a good time to have a job. Period. If you get a new job, even better given the overall economic situation. A lot of companies are instituting hiring freezes, so count yourself lucky to not only have a job offer, but one you feel is quite good. (And congratulations!)
Regardless of where you take a new job, there is always a risk that as being the new kid, should things really go sour for your new company and they need to reduce headcount that you could be one of the first to go. The risk of course is that you won't have a chance to prove yourself. Only you can weigh this up against your current situation.
HK will be affected as will other cities, but the effects will manifest themselves differently. For example, Mainland China has different exposures -- an economy that was already starting to cool down (although still booming), post-Olympics slowdown, fallout of consumer confidence following the milk scandal, etc. Some manufacturing companies are abandoning China in favour of Viet Nam and other locales, citing the rise of the yuan and that China has grown too expensive recently. However, I think Asian cities will escape relatively unscathed compared to many of their western counterparts.
While a number of banks are based in HK, keep in mind that HK is quite diversified. Until the late 90's, HK was seen as a bastion of stability in the region -- not just the gateway into China. Many companies beyond financial services have made (and continue to make) HK their base even as Singapore and Shanghai become more accessible and are seen as being on par with HK.
Personally I do not see any backlash forming in HK against expats. HK, like Shanghai and Singapore has a long history with expats that generally makes for a comfortable calm. This is not France with farmers attacking McDonalds and anyone who cannot trace their lineage back at least six centuries! (J'aime toutes les choses franais - SVP ne prennent pas l'offense si vous tes!)
Bottom line -- you have to look at your current situation and decide if you are willing to take the risk of changing jobs and moving cities. Without knowing the specifics of your situation, I would be inclined to do it -- what better a way to ride out a storm than to escape it.