The fishy section of Taste in Tuen Mun is crawling with all sorts of stuff. Stinks to high heaven. As I don't much go for seafood I have not looked that closely, but I thought they had all sorts of stuff. As for mussels - go and pick them yourself if you want them fresh! Plenty of rocks around the coast.
Definitely the wet markets. After the amazing fresh fish I've been buying I would not want anything from the supermarket. I can recommend Wan Chai wet market.
Tuen Mun Seafood Street - WOM guide
I live in Tuen Mun so this is convenient for me but I'm quite sure this is the best seafood market in Hong Kong - period.
Thats a shop at Wanchai wet market, around 60 Wanchai road, do sell live seafoods out of fish tanks. It has huge variety too, much better than wet market at Lyndhurst