The assumption that most people make is that these models can answer questions or chat with you, but in reality all they can do is take some text you provide as input and guess what the next word (or more accurately, the next token) is going to be.
These two things cannot be compared or contrasted. It's very common to see people write something like "LLMs don't actually do <thing they obviously actually do>, they just do <dismissive description of the same thing>."
Typically, like here, the dismissive description just ignores the problem of why it manages to write complete novel sentences when it's only "guessing" subword tokens, why those sentences appear to be related to the question you asked, and why they are in the form of an answer to your question instead of another question (which is what base models would do).
No you "know" what your name is and you retrieve that information. It's infinitely different to rolling a dice and picking a name based on what number comes up
I feel like I work in similar ways, if you ask my name there's a probability that I'll answer with the shorten version or the longer one pretty much randomly and there's no conscious effort about it.
While the LLMs do have hallucinations, basic stuff like this will never trigger any.
Where I feel the most differences are aren't in the token concept but rather on the deep reasoning (which we don't use as much in my opinion)
That's an interesting example, since an LLM has no concept of a "self" and literally does not know who it is. It can only answer it "correctly" if you prefix it with a prompt telling it who and what it is.
It's really not though, generally maybe but you also give it a split second to think about if it's a good time to lie, make a joke or maybe just not reveal your name.
And any of those choices will be your best guess at the next tokens in the text of the conversation respecting not only the conversation but also your self image and surroundings.
This line of reasoning that LLMs "only predict" the next token is akin to saying humans can only think or speak one word at a time. Yes, we use one token/word at a time, but it is the aggregate thought that matters, regardless of what underlies it.
I think the mistake people make is assuming that "probability" is a simple concept.
If there are 50K possible tokens and I don't have any other information, I could make a naive estimate that every token has equal probability and start generating text that is just gibberish. With the simple single-token Markov-chain example I would estimate probabilities based the previous token, and that probability estimate would be much better. If you use it for generating text it will look like something that is almost, but not quite, entirely unlike human speech. [1]
The difference lies entirely in how accurately you model the world and what information you have available when estimating probabilities. Models like GPT4 happen to be very good at it because they encode a huge amount of knowledge about the world and take a lot of context into account when estimating the probability. That's not something to be taken lightly.
I am skeptical anyone saying this is making a mistake: it only ever really comes up when someone has specific priors they're wanting to litigate - best summarized by the timeless: you cannot make a man understand something when his paycheque depends on his not understanding it.
Yeah - most of the online descriptions aren't even remotely accurate nor close to explaining how LLMs like ChatGPT actually work. They are not simple 'next-word predictors' and most of the online tutorials / info don't go into fine-tuning nor the intricate details of chain of thought reasoning (which I personally believe plays a huge role in ChatGPT's amazing performance). If you want my own detailed description you can find it here: https://photonlines.substack.com/p/intuitive-and-visual-guid...
I think too many people confuse the base model (which can be called a next token predictor) with the fine-tuned chat model which is specifically modified to carry a conversation, be helpful and be as factually correct as possible.
Comments
These two things cannot be compared or contrasted. It's very common to see people write something like "LLMs don't actually do <thing they obviously actually do>, they just do <dismissive description of the same thing>."
Typically, like here, the dismissive description just ignores the problem of why it manages to write complete novel sentences when it's only "guessing" subword tokens, why those sentences appear to be related to the question you asked, and why they are in the form of an answer to your question instead of another question (which is what base models would do).
If someone asks me "What is your name?" my reply is also simply just guessing what token would go well next in the full text of the conversation.
No you "know" what your name is and you retrieve that information. It's infinitely different to rolling a dice and picking a name based on what number comes up
This is two different parts of the system.
The LLM has this (literally - most of it is a key value store). It outputs a probability for all possible next tokens.
The sampling algorithm running the LLM then does this part, and adds randomness to make it more "creative".
So if you want factual information then don't be so random you skip over the right answer.
I feel like I work in similar ways, if you ask my name there's a probability that I'll answer with the shorten version or the longer one pretty much randomly and there's no conscious effort about it.
While the LLMs do have hallucinations, basic stuff like this will never trigger any.
Where I feel the most differences are aren't in the token concept but rather on the deep reasoning (which we don't use as much in my opinion)
Sometimes I'm tempted to think that my brain is as fuzzy if not more than an LLM
If asked repeatedly, are you gonna answer that question with a different name, depending on which "memory" you randomly pick from?
Somebody with MPD might.
That's an interesting example, since an LLM has no concept of a "self" and literally does not know who it is. It can only answer it "correctly" if you prefix it with a prompt telling it who and what it is.
You also have been told your name, most likely by your parents. It also had to be explained to you what you are, at some point in your life.
It's really not though, generally maybe but you also give it a split second to think about if it's a good time to lie, make a joke or maybe just not reveal your name.
And any of those choices will be your best guess at the next tokens in the text of the conversation respecting not only the conversation but also your self image and surroundings.
what if you're the only john in a world of steves? What would happen then?
I'd say, hi I'm john
Where can you read more on that matter of it answering your question rather than asking more?
That's called instruction tuning.
https://arxiv.org/abs/2308.10792
I came here to make this comment as well.
This line of reasoning that LLMs "only predict" the next token is akin to saying humans can only think or speak one word at a time. Yes, we use one token/word at a time, but it is the aggregate thought that matters, regardless of what underlies it.
I think the mistake people make is assuming that "probability" is a simple concept.
If there are 50K possible tokens and I don't have any other information, I could make a naive estimate that every token has equal probability and start generating text that is just gibberish. With the simple single-token Markov-chain example I would estimate probabilities based the previous token, and that probability estimate would be much better. If you use it for generating text it will look like something that is almost, but not quite, entirely unlike human speech. [1]
The difference lies entirely in how accurately you model the world and what information you have available when estimating probabilities. Models like GPT4 happen to be very good at it because they encode a huge amount of knowledge about the world and take a lot of context into account when estimating the probability. That's not something to be taken lightly.
[1] https://projects.haykranen.nl/markov/demo/
I am skeptical anyone saying this is making a mistake: it only ever really comes up when someone has specific priors they're wanting to litigate - best summarized by the timeless: you cannot make a man understand something when his paycheque depends on his not understanding it.
When the other camp is treating it like an oracle of truth and a sentient being it's hard to pick a side tbh.
Yeah, an LLM is not a Markov Chain. The only similarity is that they string words together with weighted possibilities. That's about it.
Well, it is a Markov chain if you do greedy sampling, which 99% of the time you do. So the weird part is why it still works so well.
If you do beam search, RAG, tool usage, etc then the whole system no longer is one.
Yeah - most of the online descriptions aren't even remotely accurate nor close to explaining how LLMs like ChatGPT actually work. They are not simple 'next-word predictors' and most of the online tutorials / info don't go into fine-tuning nor the intricate details of chain of thought reasoning (which I personally believe plays a huge role in ChatGPT's amazing performance). If you want my own detailed description you can find it here: https://photonlines.substack.com/p/intuitive-and-visual-guid...
I think too many people confuse the base model (which can be called a next token predictor) with the fine-tuned chat model which is specifically modified to carry a conversation, be helpful and be as factually correct as possible.