Every scheduler node has cached view of whole cluster and optimistically makes a scheduling decision, retrying on conflict?
Any tricks you did to reduce conflict rate? Is there a certain cluster saturation threshold (little free capacity) where conflict rates would get too high?
We try to spread out sandboxes evenly across the cluster (at least, across the workers which are available to take new sandboxes) to minimize conflict. But in general we don't get close to saturation thresholds so high that conflict becomes a problem, except during massive load tests. I suspect we'd see issues around 90% effective utilization.
Comments
Every scheduler node has cached view of whole cluster and optimistically makes a scheduling decision, retrying on conflict?
Any tricks you did to reduce conflict rate? Is there a certain cluster saturation threshold (little free capacity) where conflict rates would get too high?
We try to spread out sandboxes evenly across the cluster (at least, across the workers which are available to take new sandboxes) to minimize conflict. But in general we don't get close to saturation thresholds so high that conflict becomes a problem, except during massive load tests. I suspect we'd see issues around 90% effective utilization.
Makes sense thanks!