It is up to AMD, Intel and Khronos to offer APIs and tools that are actually nice to use.
They have had about 15 years to move beyond C99, stone age workflows to compile GLSL and C99 with their drivers, no libraries ecosystem, and printf debugging.
Eventually some of the issues have been fixed, after they started seeing only hardliners would put with such development experience, and then it was too late.
OneAPI builds on top of SYSCL, is basically Intel's CUDA, which it is already the second attempt to have C++ in OpenCL, during OpenCL 2.x, an effort that worked so well, that OpenCL 3.0 is basically a reboot back to OpenCL 1.0.
Also even SYSCL only got a proper kick-off after CodePlay came up with its implementation, nowadays they sell oneAPI support and tooling, after being acquired by Intel.
That is true, but that doesn't mean Nvidia is not engaging in engineering to intentionally kneecap competition. Triton and other languages like that are a huge threat and CUtile is a means to combat that threat and prevent a hardware abstraction layer.
Hundreds of thousands of developers with access to a global communication network were not stopped by AMD. Why act like dependents or wait for some bright star of consensus unless the intent is really about getting the work for free?
We don't have to wait for singular companies or foundations to fix ecosystem problems. Only the means of coordination are needed. https://prizeforge.com isn't there yet, but it is already capable of bootstrapping its own development. Matching funds, joining the team, or contributing on MuTate will all make the ball pick up speed faster.
We don't have to wait for singular companies or foundations to fix ecosystem problems.
Geohot has been working on this for about a year, and every roadblock he's encountered he has had to damn near pester Lisa Su about getting drivers fixed. If you want the CUDA replacement that would work on AMD, you need to wait on AMD. If there is a bug in the AMD microcode, you are effectively "stopped by AMD".
We have to platform and organize people, not rely on lone individuals. If there is a deep well of aligned interest, that interest needs a way to represent itself so that AMD has something to talk to, on a similar footing as a B2B relationship. When you work with other companies with hundreds and thousands of employees, it's natural that emails from individuals get drowned out or misunderstood as circulated around.
You can see in his table he calls out his AMD system as having "Good" GPU support, vs. "Great" for nvidia. So, yes, I would argue he is doing the work to platform and organize people, on a professional level to sell AMD systems in a sustainable manner - everything you claim that needs to be done and he is still bottlenecked by AMD.
A single early-stage company is not ecosystem-scale organization. It is instead the legacy benchmark to beat. This is what we do today because the best tools in our toolbox are a corporation or a foundation.
Whether AMD stands to benefit from doing more or less, we are likely in agreement that Tinygrad is a small fraction of the exposed interest and that if AMD were in conversation with a more organized, larger fraction of that interest, that AMD would do more.
I'm not defending AMD doing less. I am insisting that ecosystems can do more and that the only reason they don't is because we didn't properly analyze the problems or develop the tools.
CUDA Tile is an open source MLIR Dialect so it wouldn't take much to write MLIR transforms to map it from the Tile IR to TOSA or gpu + vector + some amdgpu or other specialty dialects.
The Tile dialect is pretty much independent of the nvidia ecosystem so all it takes is one good set of MLIR transform passes to run anything on the CUDA stack that compiles to tile out of the nvidia ecosystem prison.
So if anything this is actually a massive opportunity to escape vendor lock in if it catches on in the CUDA ecosystem.
That's not really the point. The point is that Nvidia is updating a lot of their higher level CUDA tooling to integrate with and compile to Tile IR. So this gives an escape hatch for tools built on top of CUDA to deploy outside the ecosystem.
Comments
Let's see if developers sleepwalk into another trap to keep us locked into nvidia's hardware for the next decade.
It is up to AMD, Intel and Khronos to offer APIs and tools that are actually nice to use.
They have had about 15 years to move beyond C99, stone age workflows to compile GLSL and C99 with their drivers, no libraries ecosystem, and printf debugging.
Eventually some of the issues have been fixed, after they started seeing only hardliners would put with such development experience, and then it was too late.
Isn't there OneAPI with its huge ecosystem of tools, debuggers, etc?
Yes, that is part of "it was too late".
OneAPI builds on top of SYSCL, is basically Intel's CUDA, which it is already the second attempt to have C++ in OpenCL, during OpenCL 2.x, an effort that worked so well, that OpenCL 3.0 is basically a reboot back to OpenCL 1.0.
Also even SYSCL only got a proper kick-off after CodePlay came up with its implementation, nowadays they sell oneAPI support and tooling, after being acquired by Intel.
IMO it's not Nvidia's fault the competing APIs are high friction.
AMD screwed up so badly.
That is true, but that doesn't mean Nvidia is not engaging in engineering to intentionally kneecap competition. Triton and other languages like that are a huge threat and CUtile is a means to combat that threat and prevent a hardware abstraction layer.
Hundreds of thousands of developers with access to a global communication network were not stopped by AMD. Why act like dependents or wait for some bright star of consensus unless the intent is really about getting the work for free?
We don't have to wait for singular companies or foundations to fix ecosystem problems. Only the means of coordination are needed. https://prizeforge.com isn't there yet, but it is already capable of bootstrapping its own development. Matching funds, joining the team, or contributing on MuTate will all make the ball pick up speed faster.
Geohot has been working on this for about a year, and every roadblock he's encountered he has had to damn near pester Lisa Su about getting drivers fixed. If you want the CUDA replacement that would work on AMD, you need to wait on AMD. If there is a bug in the AMD microcode, you are effectively "stopped by AMD".
We have to platform and organize people, not rely on lone individuals. If there is a deep well of aligned interest, that interest needs a way to represent itself so that AMD has something to talk to, on a similar footing as a B2B relationship. When you work with other companies with hundreds and thousands of employees, it's natural that emails from individuals get drowned out or misunderstood as circulated around.
Geohot isn't working by himself - it's part of his B2B company, tinygrad, that sells AMD systems and is VC funded.
https://tinygrad.org/#tinybox
You can see in his table he calls out his AMD system as having "Good" GPU support, vs. "Great" for nvidia. So, yes, I would argue he is doing the work to platform and organize people, on a professional level to sell AMD systems in a sustainable manner - everything you claim that needs to be done and he is still bottlenecked by AMD.
A single early-stage company is not ecosystem-scale organization. It is instead the legacy benchmark to beat. This is what we do today because the best tools in our toolbox are a corporation or a foundation.
Whether AMD stands to benefit from doing more or less, we are likely in agreement that Tinygrad is a small fraction of the exposed interest and that if AMD were in conversation with a more organized, larger fraction of that interest, that AMD would do more.
I'm not defending AMD doing less. I am insisting that ecosystems can do more and that the only reason they don't is because we didn't properly analyze the problems or develop the tools.
CUDA Tile is an open source MLIR Dialect so it wouldn't take much to write MLIR transforms to map it from the Tile IR to TOSA or gpu + vector + some amdgpu or other specialty dialects.
The Tile dialect is pretty much independent of the nvidia ecosystem so all it takes is one good set of MLIR transform passes to run anything on the CUDA stack that compiles to tile out of the nvidia ecosystem prison.
So if anything this is actually a massive opportunity to escape vendor lock in if it catches on in the CUDA ecosystem.
Yes, but why would you want to use this over the other MLIR dialects that are already cross platform?
That's not really the point. The point is that Nvidia is updating a lot of their higher level CUDA tooling to integrate with and compile to Tile IR. So this gives an escape hatch for tools built on top of CUDA to deploy outside the ecosystem.
Or it's Nvidia doing an Embrace Extend Extinguish on MLIR.
TileIR license means llvm can just fork and support it themselves as needed.
TileIR is Apache licensed so AMD can implement it as well.
Obviously they will, as with the mainframe and cloud.