I've generally found token bucket retries a lot better than circuit breakers which by nature are very bi-modal, sending no requests then floods of requests in higher scale systems. This can cause issues with memory and garbage collection when it's flapping on the order of milliseconds to seconds, and cause isues with load balancers and autoscaling when operating on the order of minutes. Token Buckets tend to dial the pressue up and down more smoothly. It's like the difference between an old fashioned thermostat and a PID loop for controlling a temperature setpoint.
- Token buckets: Proactive rate control (smooth request throttling)
They're complementary patterns. I actually have a rate limiter gem with token buckets, sliding windows, and distributed rate limiting, but I will open source it when my ship lands on a planet or a moon. Still navigating hyperspace.
Comments
I've generally found token bucket retries a lot better than circuit breakers which by nature are very bi-modal, sending no requests then floods of requests in higher scale systems. This can cause issues with memory and garbage collection when it's flapping on the order of milliseconds to seconds, and cause isues with load balancers and autoscaling when operating on the order of minutes. Token Buckets tend to dial the pressue up and down more smoothly. It's like the difference between an old fashioned thermostat and a PID loop for controlling a temperature setpoint.
Circuit breakers are inherently bi-modal (all-or-nothing) while token buckets provide smooth pressure control like cruise control vs an on/off switch.
BreakerMachines focuses specifically on failure protection rather than rate limiting. Circuit breakers and token buckets solve different problems:
- Circuit breakers: Reactive failure protection (stop calling broken services)
- Token buckets: Proactive rate control (smooth request throttling)
They're complementary patterns. I actually have a rate limiter gem with token buckets, sliding windows, and distributed rate limiting, but I will open source it when my ship lands on a planet or a moon. Still navigating hyperspace.
Thanks for the feedback