Why on earth would folks use a hair dryer to dry clothes? Bizarre.
I will admit to having done that - just to get the wet patches off my shirt when even getting to the gym involved being hot & sweaty and when the shirt hadn't dried out in the locker while I was exercising and I needed to look reasonably presentable immediately afterwards.
So long as you're not stopping anyone from using the dryer for its intended purpose I don't see a problem with that.
No, that I haven't done, although, ewww factor notwithstanding, I don't see any rational grounds to object (beyond time taken).