Locally run a ChatGPT-style LLM Trained from 800k GPT-3.5-Turbo Generations

Not exactly seeing great performance with this model. It gets tripped up by ridiculously silly questions.

For example "if you have a spoon, a knife, a fork, a carrot and a calculator, which one would you use for math?"

None. The question is grammatically incorrect as it implies that the person has all of those items in their possession at once

"The bus was going so fast it passed the racecar". What vehicle was going the fastest?"

It's impossible to tell which one went faster, as they were both traveling at high speeds

pumanoir 3y

ChatGPT and Claude also get tripped by both questions

refulgentis 3y

I think Claude is correct on the merits (Claude’s awesome in general)

Without more context about the speeds of the bus and racecar, I cannot determine which vehicle was going fastest based on the given statement. Simply saying that the bus passed the racecar does not provide enough information to compare their speeds.

CuriouslyC 3y

This is true, the question isn't inconsistent with the bus and the racecar travelling in opposite directions, it's just implied that they're going in the same direction.

cubefox 3y

GPT-4 (Bing) solves them correctly.

andriymOP3y

yeah, gonna need a few more M's of data work for that unfortunately.

rahimnathwani 3y

Repo linked in the second tweet: https://github.com/nomic-ai/gpt4all

braingenious 3y

I can’t wait for “competing with OpenAI” to get more clearly defined.

For example, could I use ChatGPT to create training data for a (non-llama) language model that I then sell per-device licenses to end users? OpenAI doesn’t sell per-device licenses, so I wouldn’t be competing with them.

Right?

Comments