So I decided to take the bus in the end.
I left on Friday evening at 7:15pm (the weekend before chinese new year) with a CTS bus - ended up taking 5 hours total. The highway was pretty clear most of the time with the bus driver going pretty fast, snaking in and out of lanes and overtaking other cars. But there were two major choke points: before the intersection for the Humen bridge, and before the tolls at the end of the highway. It took 30+ minutes to get through maybe 5km of jammed traffic in each case.
On the other hand, coming back was a breeze. With zero traffic heading towards HK, it took about 2h50min (China Hotel stop to Sheung Wan). Bus didn't stop at the other Guangzhou stops either, I guess there were no passengers to pick up. Left just after 5pm, got to Sheung Wan at 7:55pm or so. That was brilliant, comparable time with the train once you factor getting to Guangzhou East (if you're not in that neighbourhood).
Border crossing was easy and quick in both directions. Long China queues (but seemed to move fast-ish), and a short foreigner queue. Had to wait 20-30min for the second bus to depart after the border, but that's just how it works.
Overall I would do it again - obviously the train still wins for predictability and speed (most of the time), but for the price the bus is still pretty good and comfortable.