Comment on Ch.at – A lightweight LLM chat service accessible through HTTP, SSH, DNS and APIparentComments−giantrobot1yA 1B model at 2-bit quantization is about the size of the average web page anymore. With some WebGPU support you could run such a model in a browser.I'm half joking. Web pages are ludicrously fat these days.−reverius421ySomething like this? https://github.com/huggingface/transformers.js-examples/tree...−giantrobot1yThat was the exact thing I was thinking of but couldn't remember the project name. Thanks!
Comments
A 1B model at 2-bit quantization is about the size of the average web page anymore. With some WebGPU support you could run such a model in a browser.
I'm half joking. Web pages are ludicrously fat these days.
Something like this? https://github.com/huggingface/transformers.js-examples/tree...
That was the exact thing I was thinking of but couldn't remember the project name. Thanks!