What about consensus? Elections? Which server is the primary for a given repository? It also doesn't matter! There's no state and no consensus here. Any server can be the primary. All updates to the write-ahead log are synchronized with an atomic compare-and-swap (CAS) operation on S3, so it's always safe for any instance of a repository to receive a push.
Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]
I really enjoyed the article. What's great about the design outlined in it is that it builds on things that work, like leveraging S3 rather than improvising your own distributed storage system.
Of course S3 goes beyond just Amazon these days. It's a proven design and architecture for doing object storage that you find across different cloud providers that you can also self host if you need to.
This article will no doubt inspire people working on projects like forgejo, gitea, tangled, etc. that are all facing the same scaling challenges.
Comments
Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]
1: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu...
Not just durability. But read after write guarantee on a distributed object store.
How many startups and corporations built on top of S3 semantics?
Snowflake and the big lakehouses are one of them. They use S3 as giant disk in the sky.
I really enjoyed the article. What's great about the design outlined in it is that it builds on things that work, like leveraging S3 rather than improvising your own distributed storage system.
Of course S3 goes beyond just Amazon these days. It's a proven design and architecture for doing object storage that you find across different cloud providers that you can also self host if you need to.
This article will no doubt inspire people working on projects like forgejo, gitea, tangled, etc. that are all facing the same scaling challenges.
It is amazing and keeps adding new features that make it better. The CAS stuff is absolutely wonderful and only a recent addition!