I use an RTX 3090 with 24gb on it, which can comfortably fit https://huggingface.co/Qwen/Qwen3.6-27B. It's freely downloadable, so you only pay for the gfx card and the electricity. There are also plenty of models out there that work well on Apple unified memory.
But mostly I just use Claude now since I already have a max subscription. I've done hybrid setups in the past where I had Claude do the hard stuff and a local LLM do the code monkey stuff, but with the right optimizations I'm no longer hitting the Claude token ceiling.
Comments
I use an RTX 3090 with 24gb on it, which can comfortably fit https://huggingface.co/Qwen/Qwen3.6-27B. It's freely downloadable, so you only pay for the gfx card and the electricity. There are also plenty of models out there that work well on Apple unified memory.
But mostly I just use Claude now since I already have a max subscription. I've done hybrid setups in the past where I had Claude do the hard stuff and a local LLM do the code monkey stuff, but with the right optimizations I'm no longer hitting the Claude token ceiling.
Thank you !