Where are you going to live here so you "won't get wet when it rains in Hong Kong?" Ok, now think of air pollution a bit similar.
There are no air pollution monitoring stations on the Peak, or on the South Side of the Island, or in Sai Kung or Clearwater Bay, but you often see real estate agents say "the air quality is better here". Next time just say "prove it" and see what their response is. I'd love to see scientific data that shows the Peak or other areas are better, but so far it's mostly anecdotal.
Just because you can see blue skies doesn't mean the pollution is levels are safe. Just a few weeks ago we had "gorgeous" blue skies overhead (like a postcard), but the PM2.5 pollution was about 150% of the World Health Organization maximums throughout most of Hong Kong Island and Kowloon reporting stations.
Sure, there are areas that having monitoring stations that are 'better' than others, for example being away from the cars in Central, or away from the ships in the container ports can help, but when the big wind blows down from China or pushes the residue from the coal-fired power plants over the whole of Hong Kong, it can get pretty bad all over. We've seen days where remote places like Tap Mun, out on an island up by the New Territories, has had absolutely miserable air quality. For example, here is a map from six weeks ago. Everywhere in red is > 100% of the World Health Organization maximums.

If pollution is a significant issue for you, for example you have asthma, heart disease, or another respiratory disease, then the question is not "Peak vs. Midlevels" the question is "why Hong Kong?" It may be that you find the financial or career reward sufficient to overcome the potential air pollution issues, or you may, like many of us, just love living here. If you are a generally healthy normal person, you can come here and try to reduce your exposure by living in certain areas and having air purifiers and whatnot, but you do have to watch for the effects of long term exposure. Even if you live in an area that might 'seem' like it has better air, you have to remember you'll be spending a lot of time in parts like Central or Causeway Bay that do not.