For me, nothing fancy, I just added extra ram to a gaming notebook to get enough speed on answers, since it already as a good nvidia card, keeping the api open for access from another laptop I have, via api inside my local network.
I have an extra computer for services like filesharing, samba, nfs, git, firewall, etc, for instance caching the models I'm downloading with a squid proxy, so I can test several UIs downloading the same model over again. Not every UI is offering an easy way to set a single folder to store all gguf files, or it's lacking documentation.
I'm already having a lot of fun. There's people already doing much more than this. I'm more worried about integrating and gluing in a way that will become transparent after the new year, local models or not.
Also how to glue this with obsidian/logseq/neovim/etc in a way that I can use with fewest keystrokes possible, instead just uploading a gigantic context or sensible source code files.
Comments
For me, nothing fancy, I just added extra ram to a gaming notebook to get enough speed on answers, since it already as a good nvidia card, keeping the api open for access from another laptop I have, via api inside my local network.
I have an extra computer for services like filesharing, samba, nfs, git, firewall, etc, for instance caching the models I'm downloading with a squid proxy, so I can test several UIs downloading the same model over again. Not every UI is offering an easy way to set a single folder to store all gguf files, or it's lacking documentation.
I'm already having a lot of fun. There's people already doing much more than this. I'm more worried about integrating and gluing in a way that will become transparent after the new year, local models or not.
Also how to glue this with obsidian/logseq/neovim/etc in a way that I can use with fewest keystrokes possible, instead just uploading a gigantic context or sensible source code files.