A continuously running agent might require 100-500W for inference, so comparable to gaming or a small space heater. Not obscene, but also not negligible.
If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.
We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.
To serve a frontier model you need a lot more than one GPU... at least 8. But for better efficiency they batch multiple requests on one server. So it is sort of a coincidence that it ends up roughly in 1 consumer GPU range. A different architecture (e.g. linear attention / RNN) could end up with more peak power usage.
All of a sudden we are selectively squeamish with computer resource usage, when we were fine having all that fun with computers and hardware, 3 monitor setups, using graphic cards to play games (dear lord!) and tinkering around with home rigs of every proportion and wattage for no reason at all.
Driving a car consumes 25+ more energy per hour than gaming. So urban planning which encourages people to drive likely results in an order of magnitude more waste than all home computer use.
Presumably you're fine with using electricity to heat/cool your house, yes?
Would you, then, also be fine with running three extra air conditioners/space heaters that do nothing but sit outside on the lawn?
"People are OK with the energy usage already happening, so they should be OK with adding 32,767 AI datacenters too" doesn't make sense. It's essentially a reductio ad absurdum. Of course we're "selectively squeamish with computer resource usage"; some usage is obviously useful to us (with entertainment also in the "useful" category—we're not robots!), while for many, many of us, the extra datacenters are having anywhere from a neutral to a profoundly net-negative impact on our lives even before you consider the resource usage.
I mean, if you actually understood the scale of AI power usage, it would make sense. A TV can run 200 watts/hour, so this is equivalent to... watching TV?
Or, perhaps, watching TV in the background while you program.
We are not in a power crisis, and data centers have had basically no impact on our lives compared to previous technologies. There are more data centers used for the internet, but somehow you're still here, using a data center to complain about checks notes data centers? Really? Well that's a bit cheeky.
Sorry I'm a bit old school I don't watch tv when I program. What's the point of watching tv while programming anyway ? But can you really watch tv while programming ?
I can't
But maybe when you watch tv while programming you can watch documentaries about water crisis triggered by I datacenter usage, or maybe you can watch some reports about Elon using very poluting gas turbines to generate energy for their massive grok datacenter ...
I'm sorry, when individual datacenters are drawing more power than entire cities, and needing to build entire new fossil-fuel-burning power plants to service their needs, you'll excuse me if I'm skeptical of your "it's just like watching TV" excuse there.
You might find it enlightening to look at the power draw and environmental impact of other technologies, for a sense of scale.
LLMs didn't even exist when we started expressing concern about global warming, the internet still uses the vast majority of data centers and thus power, smartphone manufacturing is a nightmare on multiple levels, the environmental effects of factory farming are staggering even if you apply zero moral weight to the suffering, the existence of automobiles... what automobiles used to be like...
It just seems bizarre to care about these issues solely in the context of one particular technology, while giving a free pass to everything prior.
No one's saying we shouldn't care about the impact of other technologies and industries too.
But when the AI industry is sinking hundreds of billions of dollars into building a huge number of datacenters, many of which have already been proven to have environmental and legal problems, sitting there saying, "You didn't complain when we got TV!" or even "You didn't complain when the datacenters were for AWS!" is just transparently trying to change the subject without addressing the very real concerns.
It's hardly "changing the subject" to try and establish whether this is actually a proportional effort. Your concern should, in fact, be relative to the harm something does. A single car killing one person is not a reason to demand the abolishment of the automotive industry.
If you care about saving the environment, you could do vastly more by protesting a dozen different causes. You can't give any reason this cause is special - just vague "environmental and legal problems" that probably add up to a half-dozen anecdotes. Meanwhile, Google is out there spending hundreds of billions of dollars and racking up it's own record of proven environmental and legal problems, and you can't even bring yourself to say "yeah, and I condemn Google and all other data centers" - partly because you're using a data center to send the message in the first place
It's hardly "changing the subject" to try and establish whether this is actually a proportional effort.
No, that's basically the definition of whataboutism.
We are talking about the problems with datacenters. You are trying to claim that we shouldn't be concerned with the problems of datacenters if we aren't at least equally concerned with the problems of X, Y, and Z other things.
There is no world in which that's a sound argument.
There is a world. In that world data centers are not a net problem, but net solution.
Looking at car exhaust and concluding that cars must be bad would be a questionable conclusion. We tolerate them despite the pollution they cause, not for it.
The same can go for data centers, again, if you conclude that they are net solution. If you don't, then it won't for you. The market simply disagrees with you in that case.
Comments
Yeah let's make llm waste a whole city energy / water playing games just for fun !
The huge energy waste is from model training, not inference
A continuously running agent might require 100-500W for inference, so comparable to gaming or a small space heater. Not obscene, but also not negligible.
If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.
We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.
To serve a frontier model you need a lot more than one GPU... at least 8. But for better efficiency they batch multiple requests on one server. So it is sort of a coincidence that it ends up roughly in 1 consumer GPU range. A different architecture (e.g. linear attention / RNN) could end up with more peak power usage.
Oh really, so people are just building massive datacenters just for training right ?
"waste"
All of a sudden we are selectively squeamish with computer resource usage, when we were fine having all that fun with computers and hardware, 3 monitor setups, using graphic cards to play games (dear lord!) and tinkering around with home rigs of every proportion and wattage for no reason at all.
Driving a car consumes 25+ more energy per hour than gaming. So urban planning which encourages people to drive likely results in an order of magnitude more waste than all home computer use.
Presumably you're fine with using electricity to heat/cool your house, yes?
Would you, then, also be fine with running three extra air conditioners/space heaters that do nothing but sit outside on the lawn?
"People are OK with the energy usage already happening, so they should be OK with adding 32,767 AI datacenters too" doesn't make sense. It's essentially a reductio ad absurdum. Of course we're "selectively squeamish with computer resource usage"; some usage is obviously useful to us (with entertainment also in the "useful" category—we're not robots!), while for many, many of us, the extra datacenters are having anywhere from a neutral to a profoundly net-negative impact on our lives even before you consider the resource usage.
I mean, if you actually understood the scale of AI power usage, it would make sense. A TV can run 200 watts/hour, so this is equivalent to... watching TV?
Or, perhaps, watching TV in the background while you program.
We are not in a power crisis, and data centers have had basically no impact on our lives compared to previous technologies. There are more data centers used for the internet, but somehow you're still here, using a data center to complain about checks notes data centers? Really? Well that's a bit cheeky.
Sorry I'm a bit old school I don't watch tv when I program. What's the point of watching tv while programming anyway ? But can you really watch tv while programming ? I can't
But maybe when you watch tv while programming you can watch documentaries about water crisis triggered by I datacenter usage, or maybe you can watch some reports about Elon using very poluting gas turbines to generate energy for their massive grok datacenter ...
I'm sorry, when individual datacenters are drawing more power than entire cities, and needing to build entire new fossil-fuel-burning power plants to service their needs, you'll excuse me if I'm skeptical of your "it's just like watching TV" excuse there.
You might find it enlightening to look at the power draw and environmental impact of other technologies, for a sense of scale.
LLMs didn't even exist when we started expressing concern about global warming, the internet still uses the vast majority of data centers and thus power, smartphone manufacturing is a nightmare on multiple levels, the environmental effects of factory farming are staggering even if you apply zero moral weight to the suffering, the existence of automobiles... what automobiles used to be like...
It just seems bizarre to care about these issues solely in the context of one particular technology, while giving a free pass to everything prior.
This is whataboutism.
No one's saying we shouldn't care about the impact of other technologies and industries too.
But when the AI industry is sinking hundreds of billions of dollars into building a huge number of datacenters, many of which have already been proven to have environmental and legal problems, sitting there saying, "You didn't complain when we got TV!" or even "You didn't complain when the datacenters were for AWS!" is just transparently trying to change the subject without addressing the very real concerns.
It's hardly "changing the subject" to try and establish whether this is actually a proportional effort. Your concern should, in fact, be relative to the harm something does. A single car killing one person is not a reason to demand the abolishment of the automotive industry.
If you care about saving the environment, you could do vastly more by protesting a dozen different causes. You can't give any reason this cause is special - just vague "environmental and legal problems" that probably add up to a half-dozen anecdotes. Meanwhile, Google is out there spending hundreds of billions of dollars and racking up it's own record of proven environmental and legal problems, and you can't even bring yourself to say "yeah, and I condemn Google and all other data centers" - partly because you're using a data center to send the message in the first place
No, that's basically the definition of whataboutism.
We are talking about the problems with datacenters. You are trying to claim that we shouldn't be concerned with the problems of datacenters if we aren't at least equally concerned with the problems of X, Y, and Z other things.
There is no world in which that's a sound argument.
There is a world. In that world data centers are not a net problem, but net solution.
Looking at car exhaust and concluding that cars must be bad would be a questionable conclusion. We tolerate them despite the pollution they cause, not for it.
The same can go for data centers, again, if you conclude that they are net solution. If you don't, then it won't for you. The market simply disagrees with you in that case.
This has nothing to do with "the market".
This is a handful of incredibly wealthy companies deciding for us all.
I guess we better go ban swimming pools: they kill people! people drown in them.
What possible counter-argument can you offer to that?
I was just using the terminology of the person I was replying to
if that was true tokens would be free