It can even solve IQ tests...I mean, how much further are we moving the goal post?
Is there a model that can solve differential equations symbolically and numerically? Most of modern engineering just boils down to diff.eqs whether ordinary or partial. It's our current best method to reason about stuff and control them.
It can even solve IQ tests...I mean, how much further are we moving the goal post?
The problem with test like this is that when trained on the existing big datasets (commoncrawl etc.), chances are the test is already in the input so the validation is not proper. Its the same thing with all the "AI beats SAT" headlines. The exercises for those very tests exist all over the internet already.
It's well documented that these models can solve variations of questions that are not found anywhere in their training set, and even entirely novel problems invented by prompters. Not with 100% success, but they can do it far with a rate far better than chance, so the idea that they're pulling responses from their training data is simply not correct.
Ok maybe I should rephrase but think of it like this:
Would your IQ test score be accurate if you had a week beforehand to train solving IQ tests?
You don't get exactly the same test in the end, similar with SAT, but the constraints we put on these tests (they have to be comparable) produce patterns in the questions you can train for. This is the same logic why people can train to improve their SAT scores, if they were a measure of true innate intelligence their training would have no impact on their score.
If we flip the question around it seems more reasonable: would your IQ test score be accurate if you did not train beforehand on practice IQ tests?
I would presume not - most tests are timed, and if you are spending time on first-time only tasks in understanding the problem then the result is inaccurate. If you train out those first-time tasks so that you are repeatably using the time budget in the test to solve problems then you should reach some kind of steady state and produce repeatable and more accurate test scores.
My take is that the repeatable scores measuring your steady state in the task would be more accurate than the untrained scores with an unknown amount of initialization time within each problem. I would make a similar claim to naasking below that this could account for some of the Flynn effect.
Yes but the critical difference is that AI couldn’t pass these tests if they weren’t trained on a very similar set of questions and answers.
Not every person takes a SAT prep class to improve their test score. There are lots of people who are truly above average in terms of intelligence and can score very high on the first try.
“ Obviously they wouldn't do as well but they still do better than chance” is a huge understatement. If a model wasn’t trained with any SAT questions and answers but was trained with the same verbal and mathematical knowledge a high school student would have, the AI would do extremely poorly in an actual test. In contrast, the vast majority of human test takers would score leagues above picking answers by chance.
Your original reply insinuated that the AI is learning very similar to how humans do and that’s just not true. Yes, humans do pattern matching based on prior experiences/knowledge like AI does when you train a model, but human intelligence goes way beyond that.
Yes, humans do pattern matching based on prior experiences/knowledge like AI does when you train a model, but human intelligence goes way beyond that.
Humans are trained on orders of magnitude more multimodal data over their lifetimes. Also, humans are not borne as an unbiased model, billions of years of evolution have crafted many implicit biases into our cognition (like a propensity to language, facial recognition, etc.). All machine learning models are true blank slates, so it takes a lot more data just to build up to the same starting point as a newborn human.
All that's to say that you have no basis upon which to claim that AI learning is NOT similar to how humans do it, or that human intelligence "goes way beyond hat", it's just that humans have a head start and a lot more data to work with.
“Humans are trained on orders of magnitude more multimodal data over their lifetimes. Also, humans are not borne as an unbiased model, billions of years of evolution have crafted many implicit biases into our cognition (like a propensity to language, facial recognition, etc.).”
What in the world are you talking about? I must be talking to chatgpt and am done with this thread. We were originally discussing the differences in methodology between AI and humans for passing standardized exams. Those involve tasks like applying well-defined mathematical concepts to a brand new problem, not “multimodal” data or facial recognition.
If you don't understand what I'm talking about then you don't understand how how these transformer AIs learn and solve problems, so maybe you shouldn't opine about how AI couldn't pass these tests. Your claimed differences between how humans and AIs work is conjecture that can be explained by what I described rather than fundamental differences in how these systems work.
Here's simple example: (x-x+c)=c
gpt struggles with such examples if `x` is a large number e.g., x=123_000_000_456 and `c` some specific number,
but it is easy for humans.
It's easy for humans once they're taught what variables mean and after 10 or so years of exposure to a real world multimodal training set that's orders of magnitude more data than GPT has seen. Also, algebra is not so easy for people with IQs lower than 90, so not exactly all humans right? What exactly am I supposed infer about how GPT or other AIs and human brains operate from this apples to cars comparison?
You don't have to point out failure modes of GPT, I know what they are. The question we're discussing here is what this indicates, if anything, about how these systems operate as compared to human brains, and whether the differences come down to training data or the fundamental architecture.
Transcript of a conversation I just had with ChatGPT:
Prompt:
I'm talking with a user named "331c8c71" on Hacker News. According to statistics I found by Googling, the number of Hacker News readers living in the USA is 36%. Assuming this is correct, what is the probability that 331c8c71 does not reside in the USA? Please explain your reasoning.
Response:
If we assume that the statistic you found is accurate, then we know that 36% of Hacker News readers live in the USA. Therefore, the probability that any given Hacker News reader does not live in the USA is 100% - 36% = 64%.
However, we do not know if 331c8c71 is a typical Hacker News reader. They could be part of a demographic that is more or less likely to live in the USA. Without more information, we cannot say for certain what the probability is that 331c8c71 does not reside in the USA.
So, the answer is 64% assuming that 331c8c71 is a typical Hacker News reader, but it may not be an accurate estimate depending on additional information about the user.
---
Like I said, these models are far from 100% accurate, and there is lots they get wrong, but they clearly are capable of some kind of reasoning that goes beyond simple text substitution of training data.
The fact that just a few years ago even parsing the question would be an achievement. I feel like people are just glossing over how incredible this answer is.
I feel like there is some contingent of people who are really bent on downplaying the achievements of AI as of late. Its objectively insane, yet somehow every discussion is still sprinkled with some form of "It told me 8x8=60 so I closed the window and never used it again"
True, but the problem is that they never know. They don't interact with the real world and have no way of verifying their training data's accuracy. Perhaps they could assign a confidence level to their response? But then, if they assign a response a high confidence level and produce an incorrect answer, it will compound their failure. Not only were they wrong, but they vouched for their wrong answer.
Google search was incredibly valuable immediately even if most links could have been rubbish. I can't say the same with the current LLMs
It is an incredible achievement that LLMs produce human-like output (e.g., wouldn't know if a gpt bot answers me unless we are discussing a topic where precision/accuracy are important)
but they hallucinate (they are confident BS-generators).
The hype is that LLMs can solve any problem and replace humans (jobs). It is not so.
It may depend on what you do but I find it is easier/faster to do the work myself then to spot and fix [a possibly subtle] error in AI output. Though some of the specific things will improve in time and you can find tasks where AI is useful even today.
I don't see how the models can improve for general tasks (AGI) without being existential threat to humans (not just jobs).
Also it doesn't seem like such a leap to couple these language models with dedicated computation systems and similar. Think training a model to feed prompts to Wolfram Alpha to actually compute the results, then reporting back.
"reasoning" is a strong word. "Pattern extraction and application" is a better description of what is happening.
In particular, LLMs fail miserably at tasks like "apply this simple pattern many times in succession" aka "for-loop", because they can't count in an abstract way, only on concrete contexts.
IQ test, the sat, and exact mathematics are very different fields though. ChatGPT etc. are very good at "emulating" by sheer force and size logic, exact math doesn't work with "emulation"
I mean, how much further are we moving the goal post?
Look at it this way: humans don’t have BPE-encoded text as input to their brain. It is ALL visual input. For AGI, you would at least need to add audio input as well. And be driven by action and reward.
The learning capabilities of the brain are currently beyond the processing capabilities of current architectures. Just the notion of a model receiving only pixel data that contains a question and being able to output voice data that produces a correct answer, using no partial model trained on another corpus, is probably not tractable without significant improvements.
But the models can be very useful without being AGI!
AGI is closer to tokenization than you might think. I realized this recently when trying to do audio prediction.
There was recently a project called riffusion which generates spectrograms, then recovers audio from the spectrograms.
You might be tempted to apply this to predict speech. But speech isn’t like music. We’re communicating in language, using a sequence of tones. It’s why most speech codecs use linear predictive coding. Predicting the waveforms won’t get you anywhere; no semantic understanding of language.
So the next step up is to divide speech into a series of tones, and try to predict those sounds rather than raw waveforms.
Except… that’s literally tokenization. And there’s some evidence that this is precisely what our brains are doing.
There is definitely something symbol-adjacent that needs to happen inside of the model; this is what I assume happens in the brain. But it is not purely symbolic.
For instance, consider voicing the end of a letter: “I will definitely not be stabbed in the bac…” (where the word "back" quickly devolves into a line that crosses through the rest of the letter). It goes from symbolic to contextual, implying that the author was stabbed midway through writing it, so the voicing must end with a yell of playful agony.
The same goes for calligraphic art, such as the Al Jazeera logo, for instance, which is intended to be understood as both a sequence of Arabic letters, and a depiction of a fire. A model seeing this image for the first time, needs to see it both ways at the same time.
But it’s true that we can’t just throw a transformer at the problem, train it from scratch with video inputs and audio outputs, coupled with a sporadic reward, and suddenly have it be able to solve scans of civil engineering exams. The brain can do it, but not silicon (yet). It is easier to combine models that were trained on simpler losses (tokenized cross-entropy) on simpler problems (next-token prediction), and combine them. Not true AGI learning, but eventually it will fool people into believing it is.
I'd be cautiuous with such general statements given the rapid pace of development in this area.
Benchmark shelf lives aren't that long.
You ommitted the fact that tuning bumped it to 26% vs random.
Sure, questionable what effort is involved in that step, but at the same time, that hints to me that tuning will be the new baseline within the next 12-24 months.
Sure I would expect it to improve. But it was a bit fishy how 'it took an IQ test!' is in all the highlights but then they mumble quietly about the score that it actually got and hope no-one is listening to that bit.
Its notable that it was able to attempt it at all I suppose.
My goalpost for AGI is when Microsoft can fire their entire engineering staff, replace them with AI, and not notice any decrease in productivity or quality of output.
This test is empirically verifiable (in principle). No need to argue over whether the AI scoring X% on Y assessment task is “truly” impressive or not.
You’re confusing the goal - the goal here isn’t about the finish line but the point where people all concede that the finish line is actually reachable without any major, presently unthinkable advances.
Surely anyone familiar with software engineering knows that the finish line is reached when the finish line is reached. And no sooner.
Physicists at the beginning of the 20th century also thought that the finish line of physics was in sight and all that remained was tightening a few constants. Look how that turned out.
This perspective is too reductionist - we make predictions of success all the time based on first principles reasoning. It’s perfectly sane to try to predict, and make good arguments, if AGI is possible to achieve without new breakthroughs.
It wasn't very good at the IQ test. But yes, it is promising.
"Although there is still a large performance gap between the current model and the average level of adults, KOSMOS-1 demonstrates the potential of MLLMs to perform zero-shot nonverbal reasoning by aligning perception with language models."
Yeah, that could be good. I think LLMs will start to be really useful when they start to do math at human level. When this happens, the sky is the limit.
There's no goalpost to move. Psychologists have been saying that IQ tests are of limited scope and utility for decades. Specifically, it is widely agreed that an IQ test is not a valid way "to assess intelligence in a broader sense".
Comments
It can even solve IQ tests...I mean, how much further are we moving the goal post?
Is there a model that can solve differential equations symbolically and numerically? Most of modern engineering just boils down to diff.eqs whether ordinary or partial. It's our current best method to reason about stuff and control them.
The problem with test like this is that when trained on the existing big datasets (commoncrawl etc.), chances are the test is already in the input so the validation is not proper. Its the same thing with all the "AI beats SAT" headlines. The exercises for those very tests exist all over the internet already.
It's well documented that these models can solve variations of questions that are not found anywhere in their training set, and even entirely novel problems invented by prompters. Not with 100% success, but they can do it far with a rate far better than chance, so the idea that they're pulling responses from their training data is simply not correct.
Ok maybe I should rephrase but think of it like this: Would your IQ test score be accurate if you had a week beforehand to train solving IQ tests?
You don't get exactly the same test in the end, similar with SAT, but the constraints we put on these tests (they have to be comparable) produce patterns in the questions you can train for. This is the same logic why people can train to improve their SAT scores, if they were a measure of true innate intelligence their training would have no impact on their score.
If we flip the question around it seems more reasonable: would your IQ test score be accurate if you did not train beforehand on practice IQ tests?
I would presume not - most tests are timed, and if you are spending time on first-time only tasks in understanding the problem then the result is inaccurate. If you train out those first-time tasks so that you are repeatably using the time budget in the test to solve problems then you should reach some kind of steady state and produce repeatable and more accurate test scores.
My take is that the repeatable scores measuring your steady state in the task would be more accurate than the untrained scores with an unknown amount of initialization time within each problem. I would make a similar claim to naasking below that this could account for some of the Flynn effect.
But hasn't that always been a problem inherent to IQ tests (and SAT tests, of course)?
It is definitely one of the problems you add to a pile marked "IQ tests are meaningless and cater to the worst conceptions of human intelligence."
I mean, let's be fair. That's also how many humans learn and do IQ tests.
Yes but the critical difference is that AI couldn’t pass these tests if they weren’t trained on a very similar set of questions and answers.
Not every person takes a SAT prep class to improve their test score. There are lots of people who are truly above average in terms of intelligence and can score very high on the first try.
First, I don't think that's strictly true. Obviously they wouldn't do as well but they still do better than chance.
Second, there's evidence that this is a big part of the Flynn effect, which means humans are susceptible to a similar phenomenon.
“ Obviously they wouldn't do as well but they still do better than chance” is a huge understatement. If a model wasn’t trained with any SAT questions and answers but was trained with the same verbal and mathematical knowledge a high school student would have, the AI would do extremely poorly in an actual test. In contrast, the vast majority of human test takers would score leagues above picking answers by chance.
Your original reply insinuated that the AI is learning very similar to how humans do and that’s just not true. Yes, humans do pattern matching based on prior experiences/knowledge like AI does when you train a model, but human intelligence goes way beyond that.
Humans are trained on orders of magnitude more multimodal data over their lifetimes. Also, humans are not borne as an unbiased model, billions of years of evolution have crafted many implicit biases into our cognition (like a propensity to language, facial recognition, etc.). All machine learning models are true blank slates, so it takes a lot more data just to build up to the same starting point as a newborn human.
All that's to say that you have no basis upon which to claim that AI learning is NOT similar to how humans do it, or that human intelligence "goes way beyond hat", it's just that humans have a head start and a lot more data to work with.
“Humans are trained on orders of magnitude more multimodal data over their lifetimes. Also, humans are not borne as an unbiased model, billions of years of evolution have crafted many implicit biases into our cognition (like a propensity to language, facial recognition, etc.).”
What in the world are you talking about? I must be talking to chatgpt and am done with this thread. We were originally discussing the differences in methodology between AI and humans for passing standardized exams. Those involve tasks like applying well-defined mathematical concepts to a brand new problem, not “multimodal” data or facial recognition.
If you don't understand what I'm talking about then you don't understand how how these transformer AIs learn and solve problems, so maybe you shouldn't opine about how AI couldn't pass these tests. Your claimed differences between how humans and AIs work is conjecture that can be explained by what I described rather than fundamental differences in how these systems work.
Here's simple example: (x-x+c)=c gpt struggles with such examples if `x` is a large number e.g., x=123_000_000_456 and `c` some specific number, but it is easy for humans.
It's easy for humans once they're taught what variables mean and after 10 or so years of exposure to a real world multimodal training set that's orders of magnitude more data than GPT has seen. Also, algebra is not so easy for people with IQs lower than 90, so not exactly all humans right? What exactly am I supposed infer about how GPT or other AIs and human brains operate from this apples to cars comparison?
You don't have to point out failure modes of GPT, I know what they are. The question we're discussing here is what this indicates, if anything, about how these systems operate as compared to human brains, and whether the differences come down to training data or the fundamental architecture.
ChatGPT is not the only AI in the world.
So isn't this literally moving the goalpost? "So what an AI can beat the SAT, so can humans"
Well, my experience was that ChatGPT produced utter nonsense including literally "1/19+1/19=2/19=1" when asked a simple probability question.
Transcript of a conversation I just had with ChatGPT:
Prompt:
I'm talking with a user named "331c8c71" on Hacker News. According to statistics I found by Googling, the number of Hacker News readers living in the USA is 36%. Assuming this is correct, what is the probability that 331c8c71 does not reside in the USA? Please explain your reasoning.
Response:
If we assume that the statistic you found is accurate, then we know that 36% of Hacker News readers live in the USA. Therefore, the probability that any given Hacker News reader does not live in the USA is 100% - 36% = 64%.
However, we do not know if 331c8c71 is a typical Hacker News reader. They could be part of a demographic that is more or less likely to live in the USA. Without more information, we cannot say for certain what the probability is that 331c8c71 does not reside in the USA.
So, the answer is 64% assuming that 331c8c71 is a typical Hacker News reader, but it may not be an accurate estimate depending on additional information about the user.
---
Like I said, these models are far from 100% accurate, and there is lots they get wrong, but they clearly are capable of some kind of reasoning that goes beyond simple text substitution of training data.
The fact that just a few years ago even parsing the question would be an achievement. I feel like people are just glossing over how incredible this answer is.
I feel like there is some contingent of people who are really bent on downplaying the achievements of AI as of late. Its objectively insane, yet somehow every discussion is still sprinkled with some form of "It told me 8x8=60 so I closed the window and never used it again"
I am impressed with LLMs but I think their inability to produce an honest "I don't know" instead of hallucinating is an issue.
True, but the problem is that they never know. They don't interact with the real world and have no way of verifying their training data's accuracy. Perhaps they could assign a confidence level to their response? But then, if they assign a response a high confidence level and produce an incorrect answer, it will compound their failure. Not only were they wrong, but they vouched for their wrong answer.
Google search was incredibly valuable immediately even if most links could have been rubbish. I can't say the same with the current LLMs
It is an incredible achievement that LLMs produce human-like output (e.g., wouldn't know if a gpt bot answers me unless we are discussing a topic where precision/accuracy are important) but they hallucinate (they are confident BS-generators).
The hype is that LLMs can solve any problem and replace humans (jobs). It is not so.
It may depend on what you do but I find it is easier/faster to do the work myself then to spot and fix [a possibly subtle] error in AI output. Though some of the specific things will improve in time and you can find tasks where AI is useful even today.
I don't see how the models can improve for general tasks (AGI) without being existential threat to humans (not just jobs).
Also it doesn't seem like such a leap to couple these language models with dedicated computation systems and similar. Think training a model to feed prompts to Wolfram Alpha to actually compute the results, then reporting back.
"reasoning" is a strong word. "Pattern extraction and application" is a better description of what is happening.
In particular, LLMs fail miserably at tasks like "apply this simple pattern many times in succession" aka "for-loop", because they can't count in an abstract way, only on concrete contexts.
IQ test, the sat, and exact mathematics are very different fields though. ChatGPT etc. are very good at "emulating" by sheer force and size logic, exact math doesn't work with "emulation"
Look at it this way: humans don’t have BPE-encoded text as input to their brain. It is ALL visual input. For AGI, you would at least need to add audio input as well. And be driven by action and reward.
The learning capabilities of the brain are currently beyond the processing capabilities of current architectures. Just the notion of a model receiving only pixel data that contains a question and being able to output voice data that produces a correct answer, using no partial model trained on another corpus, is probably not tractable without significant improvements.
But the models can be very useful without being AGI!
AGI is closer to tokenization than you might think. I realized this recently when trying to do audio prediction.
There was recently a project called riffusion which generates spectrograms, then recovers audio from the spectrograms.
You might be tempted to apply this to predict speech. But speech isn’t like music. We’re communicating in language, using a sequence of tones. It’s why most speech codecs use linear predictive coding. Predicting the waveforms won’t get you anywhere; no semantic understanding of language.
So the next step up is to divide speech into a series of tones, and try to predict those sounds rather than raw waveforms.
Except… that’s literally tokenization. And there’s some evidence that this is precisely what our brains are doing.
There is definitely something symbol-adjacent that needs to happen inside of the model; this is what I assume happens in the brain. But it is not purely symbolic.
For instance, consider voicing the end of a letter: “I will definitely not be stabbed in the bac…” (where the word "back" quickly devolves into a line that crosses through the rest of the letter). It goes from symbolic to contextual, implying that the author was stabbed midway through writing it, so the voicing must end with a yell of playful agony.
The same goes for calligraphic art, such as the Al Jazeera logo, for instance, which is intended to be understood as both a sequence of Arabic letters, and a depiction of a fire. A model seeing this image for the first time, needs to see it both ways at the same time.
But it’s true that we can’t just throw a transformer at the problem, train it from scratch with video inputs and audio outputs, coupled with a sporadic reward, and suddenly have it be able to solve scans of civil engineering exams. The brain can do it, but not silicon (yet). It is easier to combine models that were trained on simpler losses (tokenized cross-entropy) on simpler problems (next-token prediction), and combine them. Not true AGI learning, but eventually it will fool people into believing it is.
Actually there's a whole new subfield called textless NLP doing just that: Learning language models from raw audio. https://ai.facebook.com/blog/textless-nlp-generating-express...
And sound, taste, smell, touch.
don't forget humor
and doom, awe, timing, a word, absurd, in, horse, common, wonder, beauty, honor, duty, doubt, danger, fear, style, worth, direction, dread, non.
https://m.youtube.com/watch?v=SKnnF5qA31I
TIL horse sense. I was thinking what on earth is the sense of horse?
phlegmatic, choleric, sanguine or melancholic?
Its crap at the visual Raven IQ test though, it scores 22% vs an algorithm that takes random guesses scoring 17%.
I'd be cautiuous with such general statements given the rapid pace of development in this area.
Benchmark shelf lives aren't that long.
You ommitted the fact that tuning bumped it to 26% vs random.
Sure, questionable what effort is involved in that step, but at the same time, that hints to me that tuning will be the new baseline within the next 12-24 months.
Sure I would expect it to improve. But it was a bit fishy how 'it took an IQ test!' is in all the highlights but then they mumble quietly about the score that it actually got and hope no-one is listening to that bit.
Its notable that it was able to attempt it at all I suppose.
Semi related, there's a (pretty good) course at OMSCS where the main project is building an agent to solve RPM problems: https://lucylabs.gatech.edu/kbai/spring-2023/project-overvie...
And quite a lot of papers about that: https://scholar.google.com/scholar?q=%22raven%27s+progressiv...
Bet you 5 bucks I can train one that gets 100%. Just gotta train it on the ravens answer key.
It's pretty big by any standards, but you may find the work of Gradshteyn and Ryzhik solves this problem nicely.
My goalpost for AGI is when Microsoft can fire their entire engineering staff, replace them with AI, and not notice any decrease in productivity or quality of output.
This test is empirically verifiable (in principle). No need to argue over whether the AI scoring X% on Y assessment task is “truly” impressive or not.
You mean that isn’t the Teams origin story?
Teams is the projection of a fourth dimension ancient Eldritch horror onto the mortal plane. AI had nothing to do with it.
You’re confusing the goal - the goal here isn’t about the finish line but the point where people all concede that the finish line is actually reachable without any major, presently unthinkable advances.
Surely anyone familiar with software engineering knows that the finish line is reached when the finish line is reached. And no sooner.
Physicists at the beginning of the 20th century also thought that the finish line of physics was in sight and all that remained was tightening a few constants. Look how that turned out.
This perspective is too reductionist - we make predictions of success all the time based on first principles reasoning. It’s perfectly sane to try to predict, and make good arguments, if AGI is possible to achieve without new breakthroughs.
It wasn't very good at the IQ test. But yes, it is promising.
"Although there is still a large performance gap between the current model and the average level of adults, KOSMOS-1 demonstrates the potential of MLLMs to perform zero-shot nonverbal reasoning by aligning perception with language models."
ChatGPT does a great job on symbolic manipulation. You have to prompt it to show derivations however vs. discussing the topic at a high level.
Yeah, that could be good. I think LLMs will start to be really useful when they start to do math at human level. When this happens, the sky is the limit.
What is human level for math? Terence Tao? Average American?
Definitely not an average American. Someone who learned to do math (proofs, application) and got fairly good at it.
There's no goalpost to move. Psychologists have been saying that IQ tests are of limited scope and utility for decades. Specifically, it is widely agreed that an IQ test is not a valid way "to assess intelligence in a broader sense".
https://en.wikipedia.org/wiki/Intelligence_quotient#Validity...
I prefer not focusing on games and benchmarks. Hopefully we ll get to robotics soon
Arnold was great in that documentary!
Roger roger.
What goalpost specifically?
Solving IQ tests which measure quantitative reasoning.
There was no movement of this goalpost.
Writing down the equations is the task for the AI.
yeah there is a model that can do differential equations