Yeah, you make some good points, orchestration has been historically painful -- we've personally seen the headaches that come with scheduling on slurm and lsf; I'd guess some of the most thorough bikeshedding in history has been around tinkering with slurm's multifactor scheduling logic. We're trying to not to re-invent the wheel with orchestration, and we're in the midst of building interfaces to hook into slurm, instead of replacing it entirely :)
As for buck2, we decided to go with it for a few reasons:
More forgiving with gradual adoption, from our experience -- running non-sandboxed actions in bazel is a pain, buck2 has been much easier to plug into existing flows.
Buck2 installation is easier, and by extension, is simpler to embed into our test runner.
Respectfully, bazel's implementation is a monolith beyond comprehension -- if we want to modify buck2, and package our own fork, we have confidence that we could do that.
Comments
Yeah, you make some good points, orchestration has been historically painful -- we've personally seen the headaches that come with scheduling on slurm and lsf; I'd guess some of the most thorough bikeshedding in history has been around tinkering with slurm's multifactor scheduling logic. We're trying to not to re-invent the wheel with orchestration, and we're in the midst of building interfaces to hook into slurm, instead of replacing it entirely :)
As for buck2, we decided to go with it for a few reasons:
More forgiving with gradual adoption, from our experience -- running non-sandboxed actions in bazel is a pain, buck2 has been much easier to plug into existing flows.
Buck2 installation is easier, and by extension, is simpler to embed into our test runner.
Respectfully, bazel's implementation is a monolith beyond comprehension -- if we want to modify buck2, and package our own fork, we have confidence that we could do that.