Skip to content

Comment on Ask HN: How to load-balance two 1 Gbps connections with a failover?

Comments

Many years ago, Fry's Electronics had a close-out on 2014 era mini-ITX AMD AM1 motherboards and CPUs (less than $50 for motherboard, CPU, heat sink and fan). I've been using them, with suitably small cases, as NAT / router / firewall / DNS machines, and even though there are just four older 2 GHz cores, with NetBSD and npf I'm able to run at full gigabit speeds just fine. I've set up around ten of these AMD systems and have run them for many years.

One way I've shared Internet with two providers is by having two routing machines, each configured with its own Internet connection, with different local IPs on the same LAN. Clients can each be configured to use the preferred gateway, or they can be switched using DHCP. Likewise, when one of the lines is down, the default route can be removed from the machine that handles that line and can be switched to the other machine's internal IP, which will DTRT and will cause clients to use the other. This has the advantage of not requiring waiting for an updated DHCP lease, so switching can be nearly instantaneous.

This is useful if one line is more robust than the other but isn't as fast - the machines that need a reliable connection always use the robust connection, and everything else opportunistically uses whatever's available.

Another is simple round-robin of NAT to both connections.

Neither helps if the line that you're using drops mid-call, but there are ways to deal with that, if you have high speed available at a datacenter. For instance, I've set up routing of a small public subnet via tinc (https://www.tinc-vpn.org) over two routing machines, each with their own Internet connection, with CARP so that packets can go through either machine. This makes handing off from one to the other transparent so that connections don't need to be reestablished.

To get back to your original query, I haven't seen any off-the-shelf NAT router that either does what a host-based router can do without some parts being proprietary or without a good number of drawbacks and limitations, nor have I seen hardware that can do anything fancy (that is, anything beyond the most simple routing / NAT) at high speeds without spending lots of $.

Recent connections that are 2 Gbps and faster are served very well by Ryzen 5600X systems with 2.5 and 10 gigabit ethernet, and those systems cost around $500 each.

Most of my machines have been set up many years ago, and automatic scripts update things like BIND and other software, so they require almost no maintenance after initial setup. On the other hand, remote administration is dead easy because they all use ssh with keys (no passwords), and can be used to help facilitate remote administration of machines on the local network, too.

In other words, I can't think of any reason any more to buy off-the-shelf NAT routers. Even if I wanted to go that route, there are too many shortcomings for me to imagine doing that - I'd give up significantly more flexibility than the amount of time I'd save would ever possibly balance.

To answer your questions about layering NAT routers, yes, you can do that, although it's discouraged. If you're forced to use the ISP's routers, you should at least ask if they can be put in bridge mode so your device can do the NAT, since many of those ISP provided routers have tiny NAT state tables and/or time out NAT states for no good reason.

Also, there's no such thing as NAT that wouldn't fit the definition of "stateful firewall", so it's hard to know what they're advertising. For instance, the Peplink Balance 580 advertises 1.5 Gbps throughput, and that's in aggregate - it definitely can't handle all five WAN at 1 Gbps at the same time, else they'd advertise 5 Gbps. Considering the prices of the hardware, a brand new, physically tiny, host-based NAT router / firewall would be both cheaper and significantly faster.

Just some thoughts :)

Neither helps if the line that you're using drops mid-call, but there are ways to deal with that, if you have high speed available at a datacenter. For instance, I've set up routing of a small public subnet via tinc (https://www.tinc-vpn.org) over two routing machines, each with their own Internet connection, with CARP so that packets can go through either machine. This makes handing off from one to the other transparent so that connections don't need to be reestablished.

Can you explain more about the setup above? I don’t understand how tinc comes into play. What OS and physical configuration are you using?

Sure :)

I'm running tinc on NetBSD.

Starting with how it works and physical setup, you have tinc running, attached to a tap interface, bridged to an ethernet interface. The ethernet interface is configured on your LAN to route (not NAT) for your local computer(s) / VoIP devices.

What's good about this is that a machine on your local network (seeable from the ethernet that's part of the bridge) can be on a 100% public address, if you want, or on a private IP range. When routing, as opposed to NAT, only the endpoints care about the existence / state of any connections, which is one of the reasons the Internet is so resilient (your intermediate hops can change without needing to renegotiate sessions).

This means that intermediate machines, including the ones running tinc, can be down for a minute - even restarted - and, if the machines are patient, the endpoint machines will just see a temporary pause.

So take two machines, each running tinc, each configured with CARP, sharing the gateway IP for the public subnet from the upstream machine. Your local machine running your video conferencing software (or, perhaps VoIP phone) communicates through the primary (active) CARP local machine. Suddenly, a shot rings out! And the primary CARP machine dies, and the maid screams, but before the scream the secondary CARP machine takes over the IP and is routing those video (or SIP) packets, so we can hear the maid scream.

There are ways to failover NAT, but they're complicated because the primary and the failover machines need to constantly share the NAT state table. Even if you're routing (not NATing) private IP subnets, so long as you're doing NAT upstream, it still works.

Does that make sense?

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.