I think you want something like Envoy in the middle to actually terminate TLS (you can still use TLS to talk to the backends, of course) and do the load balancing. Having one backend handle one HTTP/2 connection does not necessarily balance load, anyway. With Envoy in the middle, you can again load-balance requests as though they were independent connections; if a browser has an HTTP/2 connection open that wants 10 expensive things, Envoy can use 10 backends to handle that request, rather than sending 10 expensive requests to 1 backend.
The statistics and traces you get for free are also very worthwhile, not to mention more advanced things like automatic retries, canarying, outlier detection, circuit breaking, etc.
Definitely, I was simplifying a little. For that project we actually were using separate Envoy deployments per team/service. Also needed for grpc-web support, and has a json to grpc transcoder.
We could have bypassed this by having a single pool of shared Envoy instances for ingress, but at the time we wanted to avoid the complexity of multiple teams managing a single Envoy configuration. In the next year we'll hopefully switch to Istio, which will help with the multi-tenant configuration management.
Ah, thanks for sharing that story. That sounds like my environment; we started using Envoy for grpc-web... now it's in front of everything. Except, of course, it's not the main ingress controller because we had so many legacy nginx rules that we decided it was easier to use nginx as the proxy that's in front of everything. I regret it ;)
Comments
Interesting.
I think you want something like Envoy in the middle to actually terminate TLS (you can still use TLS to talk to the backends, of course) and do the load balancing. Having one backend handle one HTTP/2 connection does not necessarily balance load, anyway. With Envoy in the middle, you can again load-balance requests as though they were independent connections; if a browser has an HTTP/2 connection open that wants 10 expensive things, Envoy can use 10 backends to handle that request, rather than sending 10 expensive requests to 1 backend.
The statistics and traces you get for free are also very worthwhile, not to mention more advanced things like automatic retries, canarying, outlier detection, circuit breaking, etc.
Definitely, I was simplifying a little. For that project we actually were using separate Envoy deployments per team/service. Also needed for grpc-web support, and has a json to grpc transcoder.
We could have bypassed this by having a single pool of shared Envoy instances for ingress, but at the time we wanted to avoid the complexity of multiple teams managing a single Envoy configuration. In the next year we'll hopefully switch to Istio, which will help with the multi-tenant configuration management.
Ah, thanks for sharing that story. That sounds like my environment; we started using Envoy for grpc-web... now it's in front of everything. Except, of course, it's not the main ingress controller because we had so many legacy nginx rules that we decided it was easier to use nginx as the proxy that's in front of everything. I regret it ;)