You are making an argument against caching it at all which is clearly not what they want. So the comparison must be uncompressed storage vs compressed storage.
The compression cost is always the same. The storage cost depends on how long you keep something in cache. The decode cost is proportional to number of total hits.
So compression makes the most sense for something you want to keep a long time that will be accessed very rarely. And the least sense for something you will drop very soon and will have many people requesting it.
Maybe cloudflares workload is substantially different, but i think youre missing just how long that tail was. A “lot”, maybe half, of unique objects werent requested a second time in any meaningful period time. Like days. And edge nodes have nowhere near the iops or cycles to spend doing _any_ extra work. So yes it is a waste of resources to cache or process in any way.
Comments
You are making an argument against caching it at all which is clearly not what they want. So the comparison must be uncompressed storage vs compressed storage.
The compression cost is always the same. The storage cost depends on how long you keep something in cache. The decode cost is proportional to number of total hits.
So compression makes the most sense for something you want to keep a long time that will be accessed very rarely. And the least sense for something you will drop very soon and will have many people requesting it.
Maybe cloudflares workload is substantially different, but i think youre missing just how long that tail was. A “lot”, maybe half, of unique objects werent requested a second time in any meaningful period time. Like days. And edge nodes have nowhere near the iops or cycles to spend doing _any_ extra work. So yes it is a waste of resources to cache or process in any way.