I doubt you’re going to beat datacenter under any conditions in any model that is vaguely like for like
The comparison I saw was a small llama 8B model. ie something you can actually get usable numbers on both home and api. So something pretty commoditized
When ran 24/7, CC would possibly incur more API fees than residential electricity would cost when running on your own gear?
Claude is pretty damn expensive so plausible that you can undercut it with another model. That implies you throw out the like for like assumptions out the door though. Valid play practically, but kinda undermines the buy own rig to save argument
Comments
I doubt you’re going to beat datacenter under any conditions in any model that is vaguely like for like
The comparison I saw was a small llama 8B model. ie something you can actually get usable numbers on both home and api. So something pretty commoditized
Claude is pretty damn expensive so plausible that you can undercut it with another model. That implies you throw out the like for like assumptions out the door though. Valid play practically, but kinda undermines the buy own rig to save argument