Can recommend bus #94, through the Country Park to To Kwa Peng. Exit there and hike down the hill to cross trail to the left, maybe 2 km. Turn along that trail to Wong Shek Pier.
Pretty remote, as i recall it. Once at Wong Shek can grab ferry to Grass Island, have lunch, ferry back, return on 94.
https://wikiroutes.info/en/hong-kong?routes=55079
look at the map provided.