I use an RTX 3090 with 24gb on it, which can comfortably fit https://huggingface.co/Qwen/Qwen3.6-27B. It's freely downloadable, so you only pay for the gfx card and the electricity. There are also plenty of models out there that work well on Apple unified memory.
But mostly I just use Claude now since I already have a max subscription. I've done hybrid setups in the past where I had Claude do the hard stuff and a local LLM do the code monkey stuff, but with the right optimizations I'm no longer hitting the Claude token ceiling.
Comments
Do you pay for the model? Or run it locally? If so, which hardware do you use? Is it fast?
I use an RTX 3090 with 24gb on it, which can comfortably fit https://huggingface.co/Qwen/Qwen3.6-27B. It's freely downloadable, so you only pay for the gfx card and the electricity. There are also plenty of models out there that work well on Apple unified memory.
But mostly I just use Claude now since I already have a max subscription. I've done hybrid setups in the past where I had Claude do the hard stuff and a local LLM do the code monkey stuff, but with the right optimizations I'm no longer hitting the Claude token ceiling.
Thank you !