For something that has any expectation of fairly high uptime, I think the more important aspect here is failover (either due to actual failures, or due to maintenance) not specifically load balancing, although at this level of traffic, it's definitely helpful to be able to spread the traffic over numerous internal resources rather than relying purely on vertical scaling of the backends.
If you're able/willing to manage the LB systems yourself, generally I'd stick two balancers (running HAProxy) in front of the application servers, and do IP failover between them.
If your service uses internal IP connectivity to e.g a database or redis cluster or what have you, I'd generally have each of the balancers configured as a primary for either public or private traffic, and as a backup for the other traffic. If your service doesn't use a clustered database service or anything like it, you can obviously omit the config that balances private traffic, and it's possible this aspect is "taken care of" for you using k8s.
The exact method of IP failover is going to depend on who is hosting your machines and what their network is configured to allow (i.e. Linode previously supported VRRP, but now do not, and BGP is used instead). AFAIK none of the "standard" IP failover techniques will work in GCP/etc. Whether this is due to legitimate technical limitations of business/profit-driven decisions is left as an exercise for the reader to further analyse.
Also consider that if you're using managed k8s, the provider of that service almost certainly has a managed load balancer service. I'm not sure what you mean by "all managed load balancers will cost a ton" - Linode's balancer as an example, has a $10 monthly flat fee.
You mention "5Gbps to the service" - does that mean the data is mostly inbound, or did you mean requests result in outbound traffic averaging 5Gbps?
It's not a coincidence that GCP's load balancer will charge you, according to my calculation and their pricing, $18 per hour at 5Gbps sustained - that's the entire business model of 'big cloud' businesses. Have you looked at any providers besides GCP/AWS/Azure for hosting this service? Some guesses about what you're using from your post + reply below plus some quick calculations suggest this would be orders of magnitude cheaper to host on Linode, or a similar "not AWS-alike" provider.
Thank you for answer! Yep, I'm moving to ovh because you pay for channel, not for consumed traffic. Ovh has a load balancer, but for 0.5pb per month it will cost you 15k. Data is mostly outbound. Haproxy will help until the channel is not saturated.
EDIT
I've checked linode pricing, their lb has a flat fee because it includes 2tb of traffic :)
Sounds like you need to perhaps consider egress data costs as a high priority factor when comparing services - by my calculation 500,000 GB on Linode would be about $2.5K, even ignoring any included transfer allowance.
Comments
For something that has any expectation of fairly high uptime, I think the more important aspect here is failover (either due to actual failures, or due to maintenance) not specifically load balancing, although at this level of traffic, it's definitely helpful to be able to spread the traffic over numerous internal resources rather than relying purely on vertical scaling of the backends.
If you're able/willing to manage the LB systems yourself, generally I'd stick two balancers (running HAProxy) in front of the application servers, and do IP failover between them.
If your service uses internal IP connectivity to e.g a database or redis cluster or what have you, I'd generally have each of the balancers configured as a primary for either public or private traffic, and as a backup for the other traffic. If your service doesn't use a clustered database service or anything like it, you can obviously omit the config that balances private traffic, and it's possible this aspect is "taken care of" for you using k8s.
The exact method of IP failover is going to depend on who is hosting your machines and what their network is configured to allow (i.e. Linode previously supported VRRP, but now do not, and BGP is used instead). AFAIK none of the "standard" IP failover techniques will work in GCP/etc. Whether this is due to legitimate technical limitations of business/profit-driven decisions is left as an exercise for the reader to further analyse.
Also consider that if you're using managed k8s, the provider of that service almost certainly has a managed load balancer service. I'm not sure what you mean by "all managed load balancers will cost a ton" - Linode's balancer as an example, has a $10 monthly flat fee.
You mention "5Gbps to the service" - does that mean the data is mostly inbound, or did you mean requests result in outbound traffic averaging 5Gbps?
It's not a coincidence that GCP's load balancer will charge you, according to my calculation and their pricing, $18 per hour at 5Gbps sustained - that's the entire business model of 'big cloud' businesses. Have you looked at any providers besides GCP/AWS/Azure for hosting this service? Some guesses about what you're using from your post + reply below plus some quick calculations suggest this would be orders of magnitude cheaper to host on Linode, or a similar "not AWS-alike" provider.
Thank you for answer! Yep, I'm moving to ovh because you pay for channel, not for consumed traffic. Ovh has a load balancer, but for 0.5pb per month it will cost you 15k. Data is mostly outbound. Haproxy will help until the channel is not saturated. EDIT I've checked linode pricing, their lb has a flat fee because it includes 2tb of traffic :)
Sounds like you need to perhaps consider egress data costs as a high priority factor when comparing services - by my calculation 500,000 GB on Linode would be about $2.5K, even ignoring any included transfer allowance.
Like I said: linode will charge you about $2.5K for half a petabyte of data.
Check hetzner instead of OVH