I really enjoyed this article.
The one point I have issue with is that the dominance of object storage in today's distributed systems is very much due to economics, not technology. There's basically cheering every little step S3 takes towards a POSIX-like distributed file system like HDFS - "consistent listing of files, yeah!". Last week it was preconditions for writing files. There's still huge gymnastics needed in Iceberg/Delta to work with S3 given the lack of atomic rename.
Comments
I really enjoyed this article. The one point I have issue with is that the dominance of object storage in today's distributed systems is very much due to economics, not technology. There's basically cheering every little step S3 takes towards a POSIX-like distributed file system like HDFS - "consistent listing of files, yeah!". Last week it was preconditions for writing files. There's still huge gymnastics needed in Iceberg/Delta to work with S3 given the lack of atomic rename.