You get 20M tokens a day with a decent coding model on a $20 subscription? I am skeptical. I am further skeptical you can do that with any privacy, so odds are any discounts are a result of you selling your sessions as training data with no option to opt out.
Not saying I get 20M tokens a day, I don't know how many I get or use. My company pays for a subscription to Anthropic and OpenAI. I use the tools at work, they tell me to, it's not my personal data. (There definitely is no privacy). All I am pointing out is that most people can not run a system like that in their garages, even if they wanted to.
You are misinformed on token amounts. As one example, z.ai [0] gives you somewhere in the ballpark of 150-300 Mtok/week, so 1-2x your amounts. Plus the model has a 1M window, and will be smarter.
Fair context. Still, those prices are at least somewhat subsidized by you selling your plaintext session data which is a non-starter for me. Especially when dealing with code or data that is under NDA and not mine to sell in exchange for cheaper tokens.
Also, when I want 1M context I have 118GB of usable vram on my Strix Halo, or I can combine 4 r9700s and have 128gb and can run 1M context models like laguna or deepseek, but in practice lots of smaller sessions is better for my workflow in most cases.
Qwen 3.8 27b is smart enough that all I want is to speed-max and paralell-max on that.
Sure privacy (or legality) considerations are valid, and depending on the subscription or the API you use, you may or may not get that.
But we are not talking about the same product anymore. Your $12k homelab does not provide the same product as a $20/month subscription (to say nothing of a $200/month one). And the trade-offs of the homely may be worth it (or necessary) for you, but it may not be worth it (or even be feasible) for others.
Well at the very least lets agree that if you are going to use cloud models, there are much better options than OpenAI and Anthropic prices for most people. One does not have to give Sam or Dario money.
Personally though, I would sooner trade my car for GPUs than let a third party be in control of the tools I use to do my job.
Comments
You get 20M tokens a day with a decent coding model on a $20 subscription? I am skeptical. I am further skeptical you can do that with any privacy, so odds are any discounts are a result of you selling your sessions as training data with no option to opt out.
Not saying I get 20M tokens a day, I don't know how many I get or use. My company pays for a subscription to Anthropic and OpenAI. I use the tools at work, they tell me to, it's not my personal data. (There definitely is no privacy). All I am pointing out is that most people can not run a system like that in their garages, even if they wanted to.
You are misinformed on token amounts. As one example, z.ai [0] gives you somewhere in the ballpark of 150-300 Mtok/week, so 1-2x your amounts. Plus the model has a 1M window, and will be smarter.
[0]: https://docs.z.ai/devpack/overview#estimated-token-allowance
Fair context. Still, those prices are at least somewhat subsidized by you selling your plaintext session data which is a non-starter for me. Especially when dealing with code or data that is under NDA and not mine to sell in exchange for cheaper tokens.
Also, when I want 1M context I have 118GB of usable vram on my Strix Halo, or I can combine 4 r9700s and have 128gb and can run 1M context models like laguna or deepseek, but in practice lots of smaller sessions is better for my workflow in most cases.
Qwen 3.8 27b is smart enough that all I want is to speed-max and paralell-max on that.
Sure privacy (or legality) considerations are valid, and depending on the subscription or the API you use, you may or may not get that.
But we are not talking about the same product anymore. Your $12k homelab does not provide the same product as a $20/month subscription (to say nothing of a $200/month one). And the trade-offs of the homely may be worth it (or necessary) for you, but it may not be worth it (or even be feasible) for others.
Well at the very least lets agree that if you are going to use cloud models, there are much better options than OpenAI and Anthropic prices for most people. One does not have to give Sam or Dario money.
Personally though, I would sooner trade my car for GPUs than let a third party be in control of the tools I use to do my job.