It's affecting the quality of the web that you, as a user, are browsing.

You know what affects the quality of the web I, a user, am browsing? A bunch of damn videos I can't get to go away, a bunch of flickering GIFs, and a bunch of horrid banners. That's a very, very direct effect. The "reduced" quality is at best a second order effect.

There's also all the malvertising, the malware spread by ad networks. I grant that this, too, is a second order effect, but if we're going to worry that web quality is reduced by some ads not getting clicked, we should worry just as much about the malvertising.