Right. Even though we provide support for both OS models and custom ones, this part of the framework lacks proper maintenance. In this case less was definitely NOT more, and it's something we're aware of.
To briefly present our current solution, the idea was to mirror API behavior. We're providing a local host to which you make "calls" (symsvr). Behind the hood, that's just a wrapper around huggingface for casual and seq2seq LLMs, or a custom interface which the user has to implement for its own model (i.e. inherit from our Engine base class and implement the needed methods).
Initially, things were a little bit easier with gpt-j-6b, which we used to bootstrap, but things got a bit messier since LLaMAv2. Please expect in the near future the framework to come directly with OS models: LLaMAv2 for the neurosymbolic engine, a local indexer (we think of using Milvus), and a local embedding engine.
Comments
Right. Even though we provide support for both OS models and custom ones, this part of the framework lacks proper maintenance. In this case less was definitely NOT more, and it's something we're aware of.
To briefly present our current solution, the idea was to mirror API behavior. We're providing a local host to which you make "calls" (symsvr). Behind the hood, that's just a wrapper around huggingface for casual and seq2seq LLMs, or a custom interface which the user has to implement for its own model (i.e. inherit from our Engine base class and implement the needed methods).
Initially, things were a little bit easier with gpt-j-6b, which we used to bootstrap, but things got a bit messier since LLaMAv2. Please expect in the near future the framework to come directly with OS models: LLaMAv2 for the neurosymbolic engine, a local indexer (we think of using Milvus), and a local embedding engine.
We welcome contributions. Please make PRs.