Some people did say something like "with two tiny kids, it's safer first to pay off the mortgage", and that a 2% difference between the mortgage interest rate and the dividend isn't worth the risk.
While flat prices in HK are at their historic highest, it's the same for shares. I can easily see share prices drop 20-30%, and entering a 1-2 year period of dropping prices. Under these conditions, it might be safer to just pay off the mortgage.
I am talking about the property I live in, not an investment property, so I am not so concerned about a drop or an increase in price. If it was an investment property, then it might be a completely different story.