It is much, much faster to ship the whole HTTP request where it needs to be
If you're already routing at the HTTP level then why not just route the request based on the HTTP method? (assuming the handlers for those are properly implemented.)
Why not just route a POST request to a backend connected to a writable database? And GET to a read-only database? I don't understand how it's any kind of an optimization to be continually bouncing queries out of the read-only replicas because Postgres threw an error.
I agree with you, but I just was trying to understand the fly.io post. They also wrote:
"Most GETs are reads, but not all of them. The platform has to work for all the requests, not just the orthodox ones."
So I think it all depends on which level of the application you take the decision to gracefully fail and replay the request elsewhere. In some case, it is trivial: as soon as you see a POST/UPDATE/DELETE, because you trust your app; sometimes you may need to take the decision later, or at a lower level.
In the simplest scenario, fly.io could just forward the request to the right region, without even bothering the app server to reply with an error, but that would work only if GET requires no writes.
We can give people a library that catches Postgres readonly errors and make it a reasonably standard experience. We can't ensure that peoples' apps have good write hygiene. We _can_, though, educate people and tell them what to look for when they're trying to optimize performance.
There's also the graphql problem (and really any kind of non-rest RPC). It's somewhat rare that applications use HTTP verbs appropriately, APIs tend to bypass HTTP methods.
You can't _only_ do that, but it seems like a reasonable place to start for REST API's. I don't think there's anything to change in your product, just that the docs should recommend ways to reroute as early as possible in a request lifespan (eg, at the routing layer or before).
I'd worry about a POST route that does some expensive/slow reads/computations in the first half of the request, and then only writes at the end – lots of lost time! Would have been much better to say, hey, for this route (or for any POST route in _my_ application), please bounce to the region with the primary db instance.
Actually, while that could be done at the application level with reasonable latency, it'd probably still be better to allow the user to write some rules at the proxy layer for the "first guess".
I get what you're trying to say, but what I'm saying is that the approach you're proposing simply doesn't work. Sure, it'll route POSTs and PUTs to the write master, that part will work fine. But GETs will randomly bomb when they throw an UPDATE at their read replica. We can't predict which of those GETs will break on behalf of our users, and the one thing we're trying to avoid asking our users to do is to do complicated surgery on their applications to make them run well on us.
Like, I think it would make sense to have the feature that routes by HTTP verb! But it would be dangerous to promote that as a write-steering feature.
No, I'm saying you do both - proactively steer routes that are obviously going to write to the primary, and use the clever retry technique for everything else.
The retry technique is something you must do as a fallback, but it causes additional latency and should be avoided when you know ahead of time that the request involves a write.
Again, the application engineer should be the one responsible for writing these rules, whether in their application's routing/middleware layer (easier) or in flys proxy layer with a rules engine (faster).
Comments
If you're already routing at the HTTP level then why not just route the request based on the HTTP method? (assuming the handlers for those are properly implemented.)
Why not just route a POST request to a backend connected to a writable database? And GET to a read-only database? I don't understand how it's any kind of an optimization to be continually bouncing queries out of the read-only replicas because Postgres threw an error.
I agree with you, but I just was trying to understand the fly.io post. They also wrote:
"Most GETs are reads, but not all of them. The platform has to work for all the requests, not just the orthodox ones."
So I think it all depends on which level of the application you take the decision to gracefully fail and replay the request elsewhere. In some case, it is trivial: as soon as you see a POST/UPDATE/DELETE, because you trust your app; sometimes you may need to take the decision later, or at a lower level.
In the simplest scenario, fly.io could just forward the request to the right region, without even bothering the app server to reply with an error, but that would work only if GET requires no writes.
You got it.
We can give people a library that catches Postgres readonly errors and make it a reasonably standard experience. We can't ensure that peoples' apps have good write hygiene. We _can_, though, educate people and tell them what to look for when they're trying to optimize performance.
There's also the graphql problem (and really any kind of non-rest RPC). It's somewhat rare that applications use HTTP verbs appropriately, APIs tend to bypass HTTP methods.
You obviously can't just do that, because ordinary applications are full of GET requests that cause database updates.
You can't _only_ do that, but it seems like a reasonable place to start for REST API's. I don't think there's anything to change in your product, just that the docs should recommend ways to reroute as early as possible in a request lifespan (eg, at the routing layer or before).
I'd worry about a POST route that does some expensive/slow reads/computations in the first half of the request, and then only writes at the end – lots of lost time! Would have been much better to say, hey, for this route (or for any POST route in _my_ application), please bounce to the region with the primary db instance.
Actually, while that could be done at the application level with reasonable latency, it'd probably still be better to allow the user to write some rules at the proxy layer for the "first guess".
I get what you're trying to say, but what I'm saying is that the approach you're proposing simply doesn't work. Sure, it'll route POSTs and PUTs to the write master, that part will work fine. But GETs will randomly bomb when they throw an UPDATE at their read replica. We can't predict which of those GETs will break on behalf of our users, and the one thing we're trying to avoid asking our users to do is to do complicated surgery on their applications to make them run well on us.
Like, I think it would make sense to have the feature that routes by HTTP verb! But it would be dangerous to promote that as a write-steering feature.
No, I'm saying you do both - proactively steer routes that are obviously going to write to the primary, and use the clever retry technique for everything else.
The retry technique is something you must do as a fallback, but it causes additional latency and should be avoided when you know ahead of time that the request involves a write.
Again, the application engineer should be the one responsible for writing these rules, whether in their application's routing/middleware layer (easier) or in flys proxy layer with a rules engine (faster).