Comment on Ask HN: Is the next big thing locally running coding agents?parentComments−giwook3moI think this is the key issue with running locally hosted models.Yes, technically you can run them on 12gb vram.But should you?Realistically 64gb seems to be the current threshold for getting meaningful work done while also maintaining a large enough context window.−baigyOP3moThis will drop further with increase in intelligence density.−giwook3moIt should, which is why I said it is the current threshold.
Comments
I think this is the key issue with running locally hosted models.
Yes, technically you can run them on 12gb vram.
But should you?
Realistically 64gb seems to be the current threshold for getting meaningful work done while also maintaining a large enough context window.
This will drop further with increase in intelligence density.
It should, which is why I said it is the current threshold.