I ended up buying 2 DGX Sparks interconnected over QSFP with the intention of getting rid (as much as I could) of any cloud-based AI provider. I'm running DS4 Flash 0731 on it and some OCR models, using Oh My Pi and OpenWebUI as my main ways to interface with the agent... and from someone that has been using Claude for a long time, I can definitely say I don't need it anymore. Not for the stuff I'm doing.
I mean, this is cool and all, but 2 DGX Sparks is like $10k, right? That's a huge upfront cost to run DS4 Flash 0731 which is definitely not comparable to higher-end Claude models. It's extra egregious when you consider how much it'd cost to run DS4 Flash 0731 via an API (how long would it take to run up a $10k bill doing what you're doing?).
and from someone that has been using Claude for a long time, I can definitely say I don't need it anymore. Not for the stuff I'm doing.
I'm curious what this implies. It suggests that you were using Claude prior to this setup, so it doesn't seem like you were limited by security or local features? I can understand that someone not wanting to send their data to Anthropic (etc.) might be willing to pay for this kind of setup to accomplish that, but what was your motivation for doing this?
If it matters, if I had enough money that I could blow $10k on 2 DGX Sparks without it being a significant cost I probably would for the hell of it, so I'm not digging on you if this is ultimately what this comes down to. But, as an investment, this doesn't seem like a very good deal.
Absolutely agreed, I don't see this as an investment leading to cost-savings in the future, at all.
I see this rather as an investment in improving my capabilities and knowledge of this technology, letting me play with a 'GPU cluster', vLLM and other technologies that otherwise would require me to rent GPUs on the cloud.
I cannot stress enough how good of an instinct I think this is; many of my long term nerd decisions over the last 20 years or so have "paid off" in ways that I literally don't know how to put a money value on. Not priceless, but freeing me from so many headaches.
Offhand, these include "preferring text and my own machines for storing information that is useful to me" over "clouds" and e.g. Word docs. Also, having my own domain and email (which I pay for).
I'm thinking of, e.g. the guy that blogged for years and years and then Google just yoinked it and it was gone. Younger me was more probably more obnoxious about it, like, serves you right -- a thing I would NEVER say now -- but, still, people like me really were right all along about this sort of thing, and I daresay it would be good if everyone followed us.
I've been sending markdown documents to my non-technical coworkers for years in the hopes that they would come to appreciate the simplicity, but they insist on futzing around with Word documents and their fragile layouts. Maybe AI will finally convert everyone to the Church of Plain Text.
Sure but the GPUs in the article are from 2021 and cost ~500-600$ a pop. The OP bought 4 so that's 2k, let's say another 500$ for the rest of the chassis and you stand at $2.5k for a monster server that will burn electricity and warm your house.
Compare that to a $5k DGX than consumes much less and has the same amount of VRAM and there is a real question as whether this is worth doing at all (well aside from the cool factor).
Everyone focuses on the frontier models, and they act like everything else is useless. The gap between say, Qwen 3.8 and the frontier models is not as large as you assume, and things on the local front have made significant gains in the past year. That is WHY Anthropic, OpenAI, etc. are worried. They NEED customers to pay top dime for top performance, but if you can spend 95% less money for 95% of the performance of the latest, bleeding edge model from Anthropic, you know which one you'd pick. Most folks don't need that extra 5-10%, especially since the cheaper models will catch up anyway.
Are you sure that AI hasn't just saturated your personal benchmark? Maybe you're just not asking it to do things which showcase its full capability. Open-source models will catch up for any given use case, but the frontier is interesting because of the possibility it will keep opening up new applications.
That's a huge upfront cost to run DS4 Flash 0731 which is definitely not comparable to higher-end Claude models.
So higher-end Claude models (I've got a subscription btw) do work fine right?
What makes you think that in six months he cannot swap DS4 Flash 0731 for another model that could be equivalent to todays' top OpenAI/Anthropic models?
Or are you going to explain in six months that, after all, the top models from Anthropic from today are unfit for use?
Comments
I ended up buying 2 DGX Sparks interconnected over QSFP with the intention of getting rid (as much as I could) of any cloud-based AI provider. I'm running DS4 Flash 0731 on it and some OCR models, using Oh My Pi and OpenWebUI as my main ways to interface with the agent... and from someone that has been using Claude for a long time, I can definitely say I don't need it anymore. Not for the stuff I'm doing.
I mean, this is cool and all, but 2 DGX Sparks is like $10k, right? That's a huge upfront cost to run DS4 Flash 0731 which is definitely not comparable to higher-end Claude models. It's extra egregious when you consider how much it'd cost to run DS4 Flash 0731 via an API (how long would it take to run up a $10k bill doing what you're doing?).
I'm curious what this implies. It suggests that you were using Claude prior to this setup, so it doesn't seem like you were limited by security or local features? I can understand that someone not wanting to send their data to Anthropic (etc.) might be willing to pay for this kind of setup to accomplish that, but what was your motivation for doing this?
If it matters, if I had enough money that I could blow $10k on 2 DGX Sparks without it being a significant cost I probably would for the hell of it, so I'm not digging on you if this is ultimately what this comes down to. But, as an investment, this doesn't seem like a very good deal.
Absolutely agreed, I don't see this as an investment leading to cost-savings in the future, at all.
I see this rather as an investment in improving my capabilities and knowledge of this technology, letting me play with a 'GPU cluster', vLLM and other technologies that otherwise would require me to rent GPUs on the cloud.
I cannot stress enough how good of an instinct I think this is; many of my long term nerd decisions over the last 20 years or so have "paid off" in ways that I literally don't know how to put a money value on. Not priceless, but freeing me from so many headaches.
Offhand, these include "preferring text and my own machines for storing information that is useful to me" over "clouds" and e.g. Word docs. Also, having my own domain and email (which I pay for).
I'm thinking of, e.g. the guy that blogged for years and years and then Google just yoinked it and it was gone. Younger me was more probably more obnoxious about it, like, serves you right -- a thing I would NEVER say now -- but, still, people like me really were right all along about this sort of thing, and I daresay it would be good if everyone followed us.
It amuses me to no end that colleagues have started using markdown for notes and documents. Thanks AI.
I've been sending markdown documents to my non-technical coworkers for years in the hopes that they would come to appreciate the simplicity, but they insist on futzing around with Word documents and their fragile layouts. Maybe AI will finally convert everyone to the Church of Plain Text.
Sure but the GPUs in the article are from 2021 and cost ~500-600$ a pop. The OP bought 4 so that's 2k, let's say another 500$ for the rest of the chassis and you stand at $2.5k for a monster server that will burn electricity and warm your house.
Compare that to a $5k DGX than consumes much less and has the same amount of VRAM and there is a real question as whether this is worth doing at all (well aside from the cool factor).
Times 2, because OP said they had two of them. That's considerably more.
Each spark has as much VRAM as the server built in the article.
Everyone focuses on the frontier models, and they act like everything else is useless. The gap between say, Qwen 3.8 and the frontier models is not as large as you assume, and things on the local front have made significant gains in the past year. That is WHY Anthropic, OpenAI, etc. are worried. They NEED customers to pay top dime for top performance, but if you can spend 95% less money for 95% of the performance of the latest, bleeding edge model from Anthropic, you know which one you'd pick. Most folks don't need that extra 5-10%, especially since the cheaper models will catch up anyway.
Are you sure that AI hasn't just saturated your personal benchmark? Maybe you're just not asking it to do things which showcase its full capability. Open-source models will catch up for any given use case, but the frontier is interesting because of the possibility it will keep opening up new applications.
So higher-end Claude models (I've got a subscription btw) do work fine right?
What makes you think that in six months he cannot swap DS4 Flash 0731 for another model that could be equivalent to todays' top OpenAI/Anthropic models?
Or are you going to explain in six months that, after all, the top models from Anthropic from today are unfit for use?
Same thing for garage mechanics with a "fun car".