I find your company fascinating, having also worked on chips (and chip dev tooling) for much of my career.
But they want to work on chips, not devtools!
I have long had a gut feeling that there's an entire industry of frustrating tools specifically to keep that industry alive. I once was shocked to learn that my company had bought licenses for a tool specifically to combine multiple IP-XACT specs into one... basically just parsing several XML files and combining their data! Outrageous.
RE orchestration: It's easy-ish now since it sounds like you're starting out with (free) open source tools, but once you start looking at things like license fair-share, you might find yourself starting to build yet-another-Slurm/LSF.
Any reason for buck2 vs bazel? Bazel seems more active (O(thousands) questions on StackOverflow for Bazel vs O(hundreds) for buck).
Yeah, you make some good points, orchestration has been historically painful -- we've personally seen the headaches that come with scheduling on slurm and lsf; I'd guess some of the most thorough bikeshedding in history has been around tinkering with slurm's multifactor scheduling logic. We're trying to not to re-invent the wheel with orchestration, and we're in the midst of building interfaces to hook into slurm, instead of replacing it entirely :)
As for buck2, we decided to go with it for a few reasons:
More forgiving with gradual adoption, from our experience -- running non-sandboxed actions in bazel is a pain, buck2 has been much easier to plug into existing flows.
Buck2 installation is easier, and by extension, is simpler to embed into our test runner.
Respectfully, bazel's implementation is a monolith beyond comprehension -- if we want to modify buck2, and package our own fork, we have confidence that we could do that.
Comments
I find your company fascinating, having also worked on chips (and chip dev tooling) for much of my career.
I have long had a gut feeling that there's an entire industry of frustrating tools specifically to keep that industry alive. I once was shocked to learn that my company had bought licenses for a tool specifically to combine multiple IP-XACT specs into one... basically just parsing several XML files and combining their data! Outrageous.
RE orchestration: It's easy-ish now since it sounds like you're starting out with (free) open source tools, but once you start looking at things like license fair-share, you might find yourself starting to build yet-another-Slurm/LSF.
Any reason for buck2 vs bazel? Bazel seems more active (O(thousands) questions on StackOverflow for Bazel vs O(hundreds) for buck).
Yeah, you make some good points, orchestration has been historically painful -- we've personally seen the headaches that come with scheduling on slurm and lsf; I'd guess some of the most thorough bikeshedding in history has been around tinkering with slurm's multifactor scheduling logic. We're trying to not to re-invent the wheel with orchestration, and we're in the midst of building interfaces to hook into slurm, instead of replacing it entirely :)
As for buck2, we decided to go with it for a few reasons:
More forgiving with gradual adoption, from our experience -- running non-sandboxed actions in bazel is a pain, buck2 has been much easier to plug into existing flows.
Buck2 installation is easier, and by extension, is simpler to embed into our test runner.
Respectfully, bazel's implementation is a monolith beyond comprehension -- if we want to modify buck2, and package our own fork, we have confidence that we could do that.