This is off-topic, but are there any communities or congregations (that aren't reddit) based around locally hosted LLMs? I'm asking because while I see a bunch of projects for exposing GGML/LLaMA to OpenAI compatible interfaces, some UIs, etc, I can't really find a good community or resources for the concept in general.
I'm working on a front-end for LLMs in general, having re-implemented a working version of OpenAI's code interpreter "plugin" already within the UI (and yes, I support file uploads), and support for the wealth of third-party OpenAI plugins that don't require auth (I've been testing with the first diagram plugin I found, it works well.) I'm planning to open source it once my breaking changes slow down.
This field moves very fast, I'm looking for feedback (and essentially testers/testing data) on what people want, and looking for prompts/chat logs/guidance templates (https://github.com/microsoft/guidance) for tasks they expect to "just work" with natural language.
Instead of being limited by the monetization for ChatGPT Plus (and limited number of messages every four hours) for extensibility within a chat interface, I want to open it and free it, with a Bring-Your-Own-(optionally local)-LLM/API key setup.
The "lmg" (Local Models General) thread on 4chan's technology board /g/[0] is the premiere community communication spot for open source models, believe it or not.
Everyone from the infamous "oobabooga" to llama.cpp's Georgi Gerganov regularly hangs out in the thread.
If you have questions, you will get answers there.
The /g/ board on 4chan has a /lmg/ general that focuses on running models locally. They regularly discuss fine tuning models, quantization tech, and building apps on text-generation-webui/kobold.
You might get some interest but it's also 4chan...
You might want to try some of the discord channels connected to some of the repos. i.e. GPT4All https://github.com/nomic-ai/gpt4all scroll down for the discord link.
I'd love to hear more about your approach for getting LLMs to understand how to use plugin commands. My own experiments have not worked very well (even vanilla ChatGPT through the gpt-3.5-turbo API doesn't seem to get the concept, most of the time).
I'll do a show HN probably at the beginning of next month after this hackathon, but basically look into langchain's "tools" and the different agents they have. You don't need langchain for this at all, but it gives you the groundwork.
I saw their Code Interpreter demo on Twitter (converting an uploaded video file in a chat UI) and decided that I need that, without continuing to pay them money (because they still haven't given me access to it yet.)
So, that, and after sam a went in front of congress for the regulatory capture play, was the motivation I needed to work towards commoditizing these fuckers.
The secret sauce here with code interpreter is, well, literally a python code interpreter you can run in your browser, and it's not so secret.
Comments
This is off-topic, but are there any communities or congregations (that aren't reddit) based around locally hosted LLMs? I'm asking because while I see a bunch of projects for exposing GGML/LLaMA to OpenAI compatible interfaces, some UIs, etc, I can't really find a good community or resources for the concept in general.
I'm working on a front-end for LLMs in general, having re-implemented a working version of OpenAI's code interpreter "plugin" already within the UI (and yes, I support file uploads), and support for the wealth of third-party OpenAI plugins that don't require auth (I've been testing with the first diagram plugin I found, it works well.) I'm planning to open source it once my breaking changes slow down.
This field moves very fast, I'm looking for feedback (and essentially testers/testing data) on what people want, and looking for prompts/chat logs/guidance templates (https://github.com/microsoft/guidance) for tasks they expect to "just work" with natural language.
Instead of being limited by the monetization for ChatGPT Plus (and limited number of messages every four hours) for extensibility within a chat interface, I want to open it and free it, with a Bring-Your-Own-(optionally local)-LLM/API key setup.
The "lmg" (Local Models General) thread on 4chan's technology board /g/[0] is the premiere community communication spot for open source models, believe it or not.
Everyone from the infamous "oobabooga" to llama.cpp's Georgi Gerganov regularly hangs out in the thread.
If you have questions, you will get answers there.
[0] https://boards.4channel.org/g/#s=lmg
You know HN has gotten lackluster when 4chan is more informed.
HN is hardly the right place for an ongoing discussion
Clearly.
The /g/ board on 4chan has a /lmg/ general that focuses on running models locally. They regularly discuss fine tuning models, quantization tech, and building apps on text-generation-webui/kobold.
You might get some interest but it's also 4chan...
it's reddit, but /r/LocalLLaMA/
A name that is destined to be obsolete.
There's also r/oobabooga and r/MachineLearning
You might want to try some of the discord channels connected to some of the repos. i.e. GPT4All https://github.com/nomic-ai/gpt4all scroll down for the discord link.
I'd love to hear more about your approach for getting LLMs to understand how to use plugin commands. My own experiments have not worked very well (even vanilla ChatGPT through the gpt-3.5-turbo API doesn't seem to get the concept, most of the time).
I'll do a show HN probably at the beginning of next month after this hackathon, but basically look into langchain's "tools" and the different agents they have. You don't need langchain for this at all, but it gives you the groundwork.
I saw their Code Interpreter demo on Twitter (converting an uploaded video file in a chat UI) and decided that I need that, without continuing to pay them money (because they still haven't given me access to it yet.)
So, that, and after sam a went in front of congress for the regulatory capture play, was the motivation I needed to work towards commoditizing these fuckers.
The secret sauce here with code interpreter is, well, literally a python code interpreter you can run in your browser, and it's not so secret.
I imagine you might find your answer in the form of a discord community
https://discord.gg/dwXDHMGY for example.