If you are not looking for a long-term career in Japan I think you can have a lot of fun for two-four years. I made the mistake of thinking I was in it for the long-haul and the second-class citizen bit can get to you. Even bought a flat (and a fixer old house, long story there) both in the cat-friendly 'burbs. Was going to use the house as a cat rescue place and re-home from there (another long story). Would not consider long-term Japan without some other ties such as a Japanese spouse. Easy to get away on weekends, housing VERY VERY clean. Hallways and such always immaculate and much better overall maintenance, even in older buildings. If headed into education, salaries are all over the place from "Eikaiwa" conversation schools (low end) to international schools (higher end) and not really sure about university salaries. I made a good salary by Japanese standards, but lower than I could have earned in the US (but was looking for a very different experience). Taxes higher than HK, same level of complexity as HK (that is, very easy, when compared to doing US taxes). When I was there I had an expat policy (some loophole where I didn't join the public system) and that was cheap but if joining the health system I think would have been about 500 USD per month (it is based, in part, on your salary). So not exactly cheap but value for money if you have any medical issues or children. Mostly not worth it for me, especially since I was paying international school fees that my employer did not reimburse.
Travel anywhere within Japan or international travel (pre-pandemic) is a breeze, I didn't even mind the relatively long train out to Narita as I am a train lover. Now Haneda has more international flights so even more convenient.
Learning Japanese (took lessons my entire 6.5 stint there, intensive for the last two years) was time-consuming and pretty necessary. Need to at least get to a basic level of proficiency. Also not exactly cheap but guessing there are way more online options than when I landed in 2003. Grammar is unnatural compared to English with the verb at the end of the sentence. Particles can drive you nuts as well. In general, grammar is more complex than Chinese. Many immersion-style classes that are geared to Koreans or Chinese who plan to attend Japanese universities, run from say 8AM to 11:15 AM (I did this for the last couple of years). The other students will have the advantage, Korean grammar is quite similar and Chinese characters are the same for some words. So a bit of a struggle to keep up with classmates, lessons were fast-paced. Some evening classes for Westerners but eventually needed more than those...