For local large language models, my current setup is Ollama running on my M1 Mac Mini with 8GB of RAM, using whatever SOTA 8B model comes out. I used to have a more powerful workstation I built in 2016 with three GTX 1070s, but the capacitors were falling off, and I could not justify replacing it when Claude and ChatGPT subscriptions are more than enough for me. I plan on building a new dedicated workstation as soon as the first-mover disadvantage comes down. Today's hardware is still too early and too expensive to warrant any significant personal investment, in my opinion.
Comments
For local large language models, my current setup is Ollama running on my M1 Mac Mini with 8GB of RAM, using whatever SOTA 8B model comes out. I used to have a more powerful workstation I built in 2016 with three GTX 1070s, but the capacitors were falling off, and I could not justify replacing it when Claude and ChatGPT subscriptions are more than enough for me. I plan on building a new dedicated workstation as soon as the first-mover disadvantage comes down. Today's hardware is still too early and too expensive to warrant any significant personal investment, in my opinion.