Ask HN: How is everyone using Local LLMs?
Hello Everyone,
Would love to know how are you guys using Local LLMS on your machine
Looking for interesting usecases here
Hello Everyone,
Would love to know how are you guys using Local LLMS on your machine
Looking for interesting usecases here
Comments
I'm running ~6 small models on an oem spark to back the various features of my personal knowledge base tool. A couple of LLMs and some non-llms too
https://github.com/verdverm/gmd
I run a "bigger" model on a second spark for coding tasks with an OpenCode Go sub for big models. It's nice to have access to the best open weight as well.
I use Bionic (https://lmstudio.ai) and various local models like Qwen Coder Next to work on various tasks. Honestly, none of the models that are runnable on a typical MacBook Pro come close to the cloud-based frontier models (open or closed) but Bionic provides a nice experience and makes trying new models out trivial.
Well i have a really powerful model implanted right into my skull. When a model is so convenient, and even free to use, you'll find that you use it all the time I think