I was testing this one days ago. It seems fine to use as a base for extra finetuning, but failed hard questions that chatgpt nailed.
One example was trying to use as a assistant to beat long games, without immediate rewards.
I was trying to log and simultaneously get feedback playing Stardew Valley. gpt-3.5-turbo-1106 basically went along with me and my daughter in a coop session giving nice suggestions, sometimes with huge gaps, but easy enough to ask more about after giving more context.
Mistral 7b and 13B was basically mixing up stardew valley with WoW and Genshin Impact, even giving a lot of context about the day I was, what the npcs answered, or things that I know on how to solve a certain quest. It straight made up non existing towns (stardew valley only has one) etc, etc.
I was running the model on a separate gaming notebook, with nvidia, while playing the game on the one I'm using now.
True, and makes sense that the logic is closing in but the breadth of the data is too narrow in 7GB's to ask questions about niche topics.
Mistral hasn't released their own official 13B/30B's yet, but i'm really looking forward to what they can do.
What is crazy is that Ultrafastbert, Speculative, Jacobi, or lookahead decoding could potentially speed up by up to 80x depending on size which could make GPT-4 like models feasible on entry level macs / Phones if similar wizardry is done memory wise.
..Yes im very optimistic after the insane progress over the last months with models like Mistral, Deepseek etc.
Comments
I was testing this one days ago. It seems fine to use as a base for extra finetuning, but failed hard questions that chatgpt nailed.
One example was trying to use as a assistant to beat long games, without immediate rewards.
I was trying to log and simultaneously get feedback playing Stardew Valley. gpt-3.5-turbo-1106 basically went along with me and my daughter in a coop session giving nice suggestions, sometimes with huge gaps, but easy enough to ask more about after giving more context.
Mistral 7b and 13B was basically mixing up stardew valley with WoW and Genshin Impact, even giving a lot of context about the day I was, what the npcs answered, or things that I know on how to solve a certain quest. It straight made up non existing towns (stardew valley only has one) etc, etc.
I was running the model on a separate gaming notebook, with nvidia, while playing the game on the one I'm using now.
True, and makes sense that the logic is closing in but the breadth of the data is too narrow in 7GB's to ask questions about niche topics.
Mistral hasn't released their own official 13B/30B's yet, but i'm really looking forward to what they can do.
What is crazy is that Ultrafastbert, Speculative, Jacobi, or lookahead decoding could potentially speed up by up to 80x depending on size which could make GPT-4 like models feasible on entry level macs / Phones if similar wizardry is done memory wise.
..Yes im very optimistic after the insane progress over the last months with models like Mistral, Deepseek etc.