I agree that it rarely falls exactly the same amount as the dividend paid since, as you said, many other factors contribute to price changes daily. The dividend is but one variable thrown in that complex mix, and it’s hard to isolate. But this doesn’t mean it’s not priced in.
And I don’t think it has to be that abstract either. In a crude example, if you gave away 100k cash to charity, your net worth simultaneously drops by 100k. For some reason we don’t naturally apply the same reality to companies when they also give away cash. But when company ABC gives away 100,000,000 cash to shareholders, it’s book value is instantly worth 100,000,000 less too and should be one of the things factored in by people who want to buy the company.
I concede transaction frictions may mean the price drop may not be perfect. But if we assume your scenario is true and that stock price doesn’t fall the full amount of the dividend (again, amongst all other variables affecting price) or even at all, then what’s to stop someone from simply buying the stock just before ex-date, selling right after, and simply collect the dividend as the profit if indeed the stock price doesn’t adjust? That scenario would be exploited by traders until it eventually had to be true.