I'm a huge scheduling nerd, and the container scheduling system in this post is probably the most impactful system I've worked on. It's quite different than existing solutions, and I personally feel it's at an interesting point in the design space -- very distributed, no strong consistency anywhere, and oriented towards massive scales. Would love to hear feedback and thoughts!
Rather than a single, serialized scheduler, we run a fleet of scheduling servers which handle sandbox creation requests concurrently. To handle a creation request, a scheduling server runs a fast scheduling algorithm against in-memory cached data. The result is that scheduling scales horizontally, and looks more like load balancing than traditional container scheduling.
What does this mean? You bucket requests on some attribute and use that to route the request (or create an ordered list of routes to try)?
We make a probabilistic routing decision based on worker load and attributes of the sandbox request. I compare to a load balancer because it's essentially just forwarding an HTTP request.
Comments
I'm a huge scheduling nerd, and the container scheduling system in this post is probably the most impactful system I've worked on. It's quite different than existing solutions, and I personally feel it's at an interesting point in the design space -- very distributed, no strong consistency anywhere, and oriented towards massive scales. Would love to hear feedback and thoughts!
What does this mean? You bucket requests on some attribute and use that to route the request (or create an ordered list of routes to try)?
We make a probabilistic routing decision based on worker load and attributes of the sandbox request. I compare to a load balancer because it's essentially just forwarding an HTTP request.
Definitely interesting to read about. It seems like it has some similarities to the approach Fly.io blogged about here: https://fly.io/blog/carving-the-scheduler-out-of-our-orchest...