Signing contracts in HK as a teacher doesn't mean a whole lot. They do have lots of scare tactics, like saying you'll have to pay the remainder of your contract's salary if you quit early etc, but none of these things actually hold up in reality.
If you tell your employer your heart isn't in it anymore, they'll usually try to find a replacement for you. If they don't, you just get fired, in which case none of their threats hold up. Turn up late every day, miss classes and be rude to the parents and you'll be kicked out before the week is up. Yes, it's kind of awkward, but if you embrace it, it can be funny too. Basically, as soon as they see that you won't bring them profit, they'll cut you.
As far as cheap, but feasible places go, Sham Shui Po is one of the best spots. I cared more about low rent than anything else, so I sacrificed a lot to keep rent down. I live on a 10th floor walk up, which isn't much larger than a decent bedroom with its own bathroom and kitchenette, 5 minutes from the subway for $2.7k - typically $3.2k with bills. Choose an apartment that's a penthouse slum (illegal structure on top of a building) or illegally subdivided and you won't have to fulfill your contract either (since they can't come after you with any legal proceedings), and will usually get away with paying less deposit. I've seen flats for $3k with 0 deposit and 1/2 a month's rent for agency fee.