I also wanted to do something like this. And I had other ideas for extensions using LLM to enhence the experience on the browser. Another one I wanted to do was a focus helper that notifies you (or block you ?) when you're not in a tab related to your work. You set a goal for your current session, e.g. "I'm currently working on my backend server", and the extension reacts on tabs unrelated to that (e.g. news websites or cat videos).
I think we're gonna be close to that, there's already models running in the browser (with https://webllm.mlc.ai/) the next step is to make it more efficient.
hey we have similar ideas )
I made a focus helper thing as well: https://grgv.xyz/blog/awf/
webllm looks cool, but I would need to upgrade my laptop for it...
Exactly what I had in mind, you nailed it. With some focus on improving the UI (a list of premade prompts, or a simpler way to make them) it could become widely used. Also, I think using the OpenAI backend is a huge blocked. I wonder how a small quantized model running in the browser would perform. Sad thing is that for now WebGPU is not running on Linux' browsers. Maybe CPU inference is enough.
unfortunately only gpt-4 worked well in my experience, smaller models would work well only for blocking simple things like "cat videos page", but not for anything else less trivial.
I have another proof-of-concept where smaller model fails compared to gpt-4: https://grgv.xyz/blog/apc/
Comments
I also wanted to do something like this. And I had other ideas for extensions using LLM to enhence the experience on the browser. Another one I wanted to do was a focus helper that notifies you (or block you ?) when you're not in a tab related to your work. You set a goal for your current session, e.g. "I'm currently working on my backend server", and the extension reacts on tabs unrelated to that (e.g. news websites or cat videos).
I think we're gonna be close to that, there's already models running in the browser (with https://webllm.mlc.ai/) the next step is to make it more efficient.
hey we have similar ideas ) I made a focus helper thing as well: https://grgv.xyz/blog/awf/ webllm looks cool, but I would need to upgrade my laptop for it...
Exactly what I had in mind, you nailed it. With some focus on improving the UI (a list of premade prompts, or a simpler way to make them) it could become widely used. Also, I think using the OpenAI backend is a huge blocked. I wonder how a small quantized model running in the browser would perform. Sad thing is that for now WebGPU is not running on Linux' browsers. Maybe CPU inference is enough.
unfortunately only gpt-4 worked well in my experience, smaller models would work well only for blocking simple things like "cat videos page", but not for anything else less trivial. I have another proof-of-concept where smaller model fails compared to gpt-4: https://grgv.xyz/blog/apc/