This is a cool demo! It does a good job hinting at how immersive the experience can be.
I'm excited about large language models for gaming. The first thing I tried with ChatGPT was to run a text-based RPG game, and it was fun the very first attempt. I imagine we'll have (a text-based) holodeck/Westworld soon enough.
In the demo I ran out of tokens pretty quickly, after talking to 3 of the characters. I wonder how the economics of this works - I'm guessing token markup?
I wrote a library for structured inference in-browser[1] and recently started building a murder-mystery demo as well[2] (very rough WIP, not yet playable).
The library might make a good backend for your project. Local llm in-browser means it's way easier to distribute since there's no need to install anything. Structured inference lets you get consistent responses out of even smaller models so you can always get a conversation, or an action json. And of course, no token limits since it runs client-side.
I wonder how the economics of this works - I'm guessing token markup?
We're mostly thinking about building tools now for a future in which many gamers will have the compute to locally run a sizable ~30B model. It's hard to me to see LLM gaming taking off in the very near term due to the compute costs for any model of reasonable immersiveness.
I wrote a library for structured inference in-browser[1]
Very cool! Excited to see where this ends up. Yeah, OpenAI is a bit annoying since you can't ask for JSON or the logit distribution. But GPT3.5-turbo right now is even cheaper than running your own llama2-70b so we stuck with that for the demo since most people probably don't have sufficient compute for this.
I've actually been able to get pretty remarkable results out of Llama 2 7B on just my laptop gpu (8gb). Just don't give it any logic puzzles. I wonder if a 13B (more or less runnable with consumer GPU hardware) fine-tuned on a gaming data set could actually be pretty immersive already?
And good luck! Will be keeping an eye on the gaming framework.
Comments
This is a cool demo! It does a good job hinting at how immersive the experience can be.
I'm excited about large language models for gaming. The first thing I tried with ChatGPT was to run a text-based RPG game, and it was fun the very first attempt. I imagine we'll have (a text-based) holodeck/Westworld soon enough.
In the demo I ran out of tokens pretty quickly, after talking to 3 of the characters. I wonder how the economics of this works - I'm guessing token markup?
I wrote a library for structured inference in-browser[1] and recently started building a murder-mystery demo as well[2] (very rough WIP, not yet playable).
The library might make a good backend for your project. Local llm in-browser means it's way easier to distribute since there's no need to install anything. Structured inference lets you get consistent responses out of even smaller models so you can always get a conversation, or an action json. And of course, no token limits since it runs client-side.
[1] https://github.com/gsuuon/ad-llama [2] https://github.com/gsuuon/ad-llama/tree/murder-mystery-solid...
Thanks!
We're mostly thinking about building tools now for a future in which many gamers will have the compute to locally run a sizable ~30B model. It's hard to me to see LLM gaming taking off in the very near term due to the compute costs for any model of reasonable immersiveness.
Very cool! Excited to see where this ends up. Yeah, OpenAI is a bit annoying since you can't ask for JSON or the logit distribution. But GPT3.5-turbo right now is even cheaper than running your own llama2-70b so we stuck with that for the demo since most people probably don't have sufficient compute for this.
I've actually been able to get pretty remarkable results out of Llama 2 7B on just my laptop gpu (8gb). Just don't give it any logic puzzles. I wonder if a 13B (more or less runnable with consumer GPU hardware) fine-tuned on a gaming data set could actually be pretty immersive already?
And good luck! Will be keeping an eye on the gaming framework.