$12k for 6 4090 for 144GB GDDR vs $20k H100 PCIe 80GB HBM2 (price likely dropping later this year when B100 is released).
And H100 has a lot of features like async and loading directly to tensor cores not present in consumer cards.
I want to root for the little guy, but it seems the AI hardware landscape will be Nvidia for the next few years. And us GPU poors accessing it via cloud (shudders).
Comments
$12k for 6 4090 for 144GB GDDR vs $20k H100 PCIe 80GB HBM2 (price likely dropping later this year when B100 is released).
And H100 has a lot of features like async and loading directly to tensor cores not present in consumer cards.
I want to root for the little guy, but it seems the AI hardware landscape will be Nvidia for the next few years. And us GPU poors accessing it via cloud (shudders).