Nvidia ships a new architecture roughly every year, and the competitive life of a given generation is plausibly two to three years, while hyperscalers depreciate the hardware over five to six
I keep seeing this take about how GPUs are going to lose value and yet somehow they don't? Why is it that A100s (released in 2020) are still in high demand and fetching high prices? E.g. RunPod is still charging $1.40-$1.60/hr for A100s and making money. Lambda Labs is charging $2.00-$2.80/hr for the same GPU.
The arguments I've heard around GPUs losing value is based on resale. So long as the first party can utilize or lease them at a high rate, it's not a risk. The idea is you don't know the environment they were used in or how hard they've been abused. The first party knows this, so long as they can continue to lease them without suffering anomalous failure rates the financials are OK.
The lingering question is something like "Coreweave declares bankruptcy, how do they liquidate all these GPUs?"
Nvidia still manufacture the A100. The argument I hear isn't only about computing power value of older gen GPUs but about their 3-5 life expectancy under continuous use.
Comments
I keep seeing this take about how GPUs are going to lose value and yet somehow they don't? Why is it that A100s (released in 2020) are still in high demand and fetching high prices? E.g. RunPod is still charging $1.40-$1.60/hr for A100s and making money. Lambda Labs is charging $2.00-$2.80/hr for the same GPU.
The arguments I've heard around GPUs losing value is based on resale. So long as the first party can utilize or lease them at a high rate, it's not a risk. The idea is you don't know the environment they were used in or how hard they've been abused. The first party knows this, so long as they can continue to lease them without suffering anomalous failure rates the financials are OK.
The lingering question is something like "Coreweave declares bankruptcy, how do they liquidate all these GPUs?"
Nvidia still manufacture the A100. The argument I hear isn't only about computing power value of older gen GPUs but about their 3-5 life expectancy under continuous use.
apart from GPUs, what about memory?