I’m planning to deploy a 1B model, feed it all the documents I’ve ever written, host it on a $149 mini-PC in my bedroom, and enable you to chat with it.
I’ve released similar projects before.
I’ll drop a post about my plans in the coming days and I’ll build and document it about two weeks later if there’s enough interest.
Comments
I see this is using GPT4o, any plans for something more sustainable? Would be interesting to see an https://openfreemap.org for LLMs.
Perhaps via an RNN like in https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-2
Or even just leverage huggingface gradio spaces? (most are Gradio apps that expose APIs https://www.gradio.app/guides/view-api-page)
I wonder if a 1B model could be close to free to host. That's an eventuality, but I wonder how long it'll take for that to be real.
I’m planning to deploy a 1B model, feed it all the documents I’ve ever written, host it on a $149 mini-PC in my bedroom, and enable you to chat with it.
I’ve released similar projects before.
I’ll drop a post about my plans in the coming days and I’ll build and document it about two weeks later if there’s enough interest.
joeldare.com
The post about my plan is now online:
https://joeldare.com/my_plan_to_build_an_ai_chat_bot_in_my_b...
Sounds cool!
A 1B model at 2-bit quantization is about the size of the average web page anymore. With some WebGPU support you could run such a model in a browser.
I'm half joking. Web pages are ludicrously fat these days.
Something like this? https://github.com/huggingface/transformers.js-examples/tree...
That was the exact thing I was thinking of but couldn't remember the project name. Thanks!