While I am 100% with you that we've arrived at the place local GPUs can do real, good work and that it helps combat the energy arguments against certain LLM usage, Drew is not likely to agree with you. I can't find it on a quick search which tells me I'm probably remembering it from his prior Mastodon account, but I distinctly remember Drew once talking about how gaming GPUs were part of the (climate) problem and that we should all take notes from the Nintendo Switch in terms of gaming joy delivered per watt.
This is not to say I disagree with Drew's desire for efficiency, even in gaming. This is simply to say: from my recollection of that blog/toot thread/whatever it was, even home LLM usage won't meet what I recall his energy bar to be, until we've got planning and orchestrator models that fit on a non-flagship phone.
I wouldn't consider this particularly hypocritical for even most people opposed to computing at the gigawatt scale, but for someone who believes a 3060 is over the line it's another story.
Comments
While I am 100% with you that we've arrived at the place local GPUs can do real, good work and that it helps combat the energy arguments against certain LLM usage, Drew is not likely to agree with you. I can't find it on a quick search which tells me I'm probably remembering it from his prior Mastodon account, but I distinctly remember Drew once talking about how gaming GPUs were part of the (climate) problem and that we should all take notes from the Nintendo Switch in terms of gaming joy delivered per watt.
This is not to say I disagree with Drew's desire for efficiency, even in gaming. This is simply to say: from my recollection of that blog/toot thread/whatever it was, even home LLM usage won't meet what I recall his energy bar to be, until we've got planning and orchestrator models that fit on a non-flagship phone.
This seems really odd given that Drew chose to write and continues to maintain Sourcehut in a language that is one to two orders of magnitude less energy-efficient than C (https://www.sciencedirect.com/science/article/pii/S016764232...)
I wouldn't consider this particularly hypocritical for even most people opposed to computing at the gigawatt scale, but for someone who believes a 3060 is over the line it's another story.