As more and more sites join, cache hits become more likely, making all sites load faster as a result.

This was not really true back when browsers allowed sharing caches between domains because of different versions and fragmentation of CDNs, and is definitely not true now when a.com and b.com would not share a cache for skypack to avoid fingerprinting.