With enough effort and priming you can trick _people_ in to believing things which are clearly untrue. Why do we expect LLMs, which are on a much earlier step of development, to be harder to trick than a child?
LLMs at the moment are really advanced autocomplete - they can fill in the next step of conversation, but they don't understand the question and respond with abstract reasoning. Yet.
they don't understand the question and respond with abstract reasoning. Yet.
What makes you think LLM's as a class of technology will ever have the capacity to really do this. I thought that no matter how big a model gets it's never actually 'thinking'.
All those prompts like 'think step by step' are just helpers along the way, because as you say it's 'really advanced autocomplete'
It depends what exactly you mean by "LLM". But an ANN is effectively a function approximator. If you made one big enough to very closely approximate the entire quantum state of a person interacting with an environment, would you still declare that nothing it could do is "thinking"?
This is silly, that's like talking about building a fusion reactor modeled after the sun. It is easy to propose something like that, but we always seem to be 10 years away from realizing it. In fact it could be easier to solve the fusion problem than trying to build a machine/software that closely approximates a human brain as you suggest.
Yes it would be wonderful if sci-fi was real, but we need to deal in what is possible in reality.
Yes, that was obviously an extreme example. But we know that it is possible in reality to implement a physical system that does what we call thinking. There is, I think, no particular reason to suppose that it's physically impossible re-implement the functionality with much less meat. Supposing that you've done this, you then just need to more clearly define "thinking" and "LLM" to determine whether changing that re-implementation to be closer to an LLM results in it losing the ability to think before it gets there.
Efficiency has everything to do with it. To run your original thought experiment would likely take more computing capacity than humanity will ever produce with current silicon based processors.
It could be that we can "get there" with much rougher approximation, but that is by no means a given. If "getting there" does require a much more complex model that actually involves simulating human neurons with much more accuracy and precision, the computing power and energy needed, again based on semiconductor computers, might simply be intractable.
I'm no more suggesting that you actually attempt to build an ANN to run a full quantum simulation of a person and their house than Einstein was suggesting that you go out and board a train that goes the speed of light. The reason it's called a thought experiment instead of a project proposal is that actually doing the thing isn't the point. It's about examining the consequences of a premise.
The difference between your thoughts experiment and Einstein's is that Einstein's had some testable implications. Yours is closer to belief in a teacup drifting in deep space.
The entire premise is unfalsifiable because it requires construction of an impossible thing:
If you made one big enough to very closely approximate the entire quantum state of a person interacting with an environment, would you still declare that nothing it could do is "thinking"?
Perhaps the question was not fully understood. A function is simply a mapping from one state into another. In principle we can define it over what ever state. As such we can consider a human thinking as kind of a function.
ANN are function effectively approximators.
The question was - if an ANN would very closely approximate the "human thinking" function then can we still say that it is not thinking?
In order to train an ANN to approximate the "human thinking" function, we would have to know that function well enough to give it examples and counterexamples. Currently we are only training LLMs to approximate the "human blabbing" function.
Imagine a human brain that has only read all what ANN has without any connection to the real world (no senses, mind you!). Would the output of this human being differ from "human blabbing"?
I don't know. This feels like an unfair question to ask. You've proposed a basically impossible engineering problem and then declared that the outcome will obviously be the outcome that you want it to be.
What if I answer, "No because humans have supernatural souls."
You can easily answer, "Souls don't exist, therefore I'm still right."
But then we actually build your machine and it turns out that Penrose's non-computable microtubules really exist and the machine is useless.
We can't know the result of such an ambitious endeavor before we go through with it. So it doesn't make much sense to me to use such a thought experiment as evidence for something much lesser that is currently contested.
You think it's unfair to ask whether a simulation of a person can think in a discussion on whether some particular class of algorithm can think? Lacking a clear definition of what exactly an LLM is and what thinking is, I can't think of a single more germane question (aside from what those definitions are, I suppose).
To approximate a function with a loop you would need a close to infinitely large neural net. Humans do have loops in their thinking, we need a new architecture for LLMs to be able to think in loops.
But we don't know how to train those to be good yet. It would require some novel step there, and it is unclear if it would still be anything like current LLMs after that.
Yeah, LLM + Reasoning Module is only as good as its weakest part. And we had more than one AI winter around the development of reasoning modules. I think you're 2x correct -- we need both and the exercise to create it is worthy of :p
With enough effort and priming you can trick _people_ in to believing things which are clearly untrue.
I think that's overstatement. The most I can find is references to making people more credulous to obscure claims ("Basketball became an Olympic discipline in 1925.") whose truth they couldn't easily discover (especially pre-Internet) [1].
There are other where a person is confronted by shills making claims and otherwise experiences more manipulation than just being exposed to text. But that seems of a different category.
Comments
With enough effort and priming you can trick _people_ in to believing things which are clearly untrue. Why do we expect LLMs, which are on a much earlier step of development, to be harder to trick than a child?
LLMs at the moment are really advanced autocomplete - they can fill in the next step of conversation, but they don't understand the question and respond with abstract reasoning. Yet.
What makes you think LLM's as a class of technology will ever have the capacity to really do this. I thought that no matter how big a model gets it's never actually 'thinking'.
All those prompts like 'think step by step' are just helpers along the way, because as you say it's 'really advanced autocomplete'
It depends what exactly you mean by "LLM". But an ANN is effectively a function approximator. If you made one big enough to very closely approximate the entire quantum state of a person interacting with an environment, would you still declare that nothing it could do is "thinking"?
This is silly, that's like talking about building a fusion reactor modeled after the sun. It is easy to propose something like that, but we always seem to be 10 years away from realizing it. In fact it could be easier to solve the fusion problem than trying to build a machine/software that closely approximates a human brain as you suggest.
Yes it would be wonderful if sci-fi was real, but we need to deal in what is possible in reality.
Yes, that was obviously an extreme example. But we know that it is possible in reality to implement a physical system that does what we call thinking. There is, I think, no particular reason to suppose that it's physically impossible re-implement the functionality with much less meat. Supposing that you've done this, you then just need to more clearly define "thinking" and "LLM" to determine whether changing that re-implementation to be closer to an LLM results in it losing the ability to think before it gets there.
I think you underestimate the efficiency of meat.
Efficiency has absolutely nothing to do with it.
Efficiency has everything to do with it. To run your original thought experiment would likely take more computing capacity than humanity will ever produce with current silicon based processors.
It could be that we can "get there" with much rougher approximation, but that is by no means a given. If "getting there" does require a much more complex model that actually involves simulating human neurons with much more accuracy and precision, the computing power and energy needed, again based on semiconductor computers, might simply be intractable.
I'm no more suggesting that you actually attempt to build an ANN to run a full quantum simulation of a person and their house than Einstein was suggesting that you go out and board a train that goes the speed of light. The reason it's called a thought experiment instead of a project proposal is that actually doing the thing isn't the point. It's about examining the consequences of a premise.
The difference between your thoughts experiment and Einstein's is that Einstein's had some testable implications. Yours is closer to belief in a teacup drifting in deep space.
And what's the unfalsifiable assertion you claim I've made?
The entire premise is unfalsifiable because it requires construction of an impossible thing:
Perhaps the question was not fully understood. A function is simply a mapping from one state into another. In principle we can define it over what ever state. As such we can consider a human thinking as kind of a function.
ANN are function effectively approximators.
The question was - if an ANN would very closely approximate the "human thinking" function then can we still say that it is not thinking?
In order to train an ANN to approximate the "human thinking" function, we would have to know that function well enough to give it examples and counterexamples. Currently we are only training LLMs to approximate the "human blabbing" function.
Imagine a human brain that has only read all what ANN has without any connection to the real world (no senses, mind you!). Would the output of this human being differ from "human blabbing"?
I don't know. This feels like an unfair question to ask. You've proposed a basically impossible engineering problem and then declared that the outcome will obviously be the outcome that you want it to be.
What if I answer, "No because humans have supernatural souls."
You can easily answer, "Souls don't exist, therefore I'm still right."
But then we actually build your machine and it turns out that Penrose's non-computable microtubules really exist and the machine is useless.
We can't know the result of such an ambitious endeavor before we go through with it. So it doesn't make much sense to me to use such a thought experiment as evidence for something much lesser that is currently contested.
You think it's unfair to ask whether a simulation of a person can think in a discussion on whether some particular class of algorithm can think? Lacking a clear definition of what exactly an LLM is and what thinking is, I can't think of a single more germane question (aside from what those definitions are, I suppose).
To approximate a function with a loop you would need a close to infinitely large neural net. Humans do have loops in their thinking, we need a new architecture for LLMs to be able to think in loops.
I don't think anyone would say that being an RNN disqualifies an architecture from being considered an LLM.
But we don't know how to train those to be good yet. It would require some novel step there, and it is unclear if it would still be anything like current LLMs after that.
In principle you could attach a reasoning module to the neural network and only use the LLM part of the network for input/output.
The design of such a reasoning module is an exercise left to the reader, obviously :p
Yeah, LLM + Reasoning Module is only as good as its weakest part. And we had more than one AI winter around the development of reasoning modules. I think you're 2x correct -- we need both and the exercise to create it is worthy of :p
With enough effort and priming you can trick _people_ in to believing things which are clearly untrue.
I think that's overstatement. The most I can find is references to making people more credulous to obscure claims ("Basketball became an Olympic discipline in 1925.") whose truth they couldn't easily discover (especially pre-Internet) [1].
There are other where a person is confronted by shills making claims and otherwise experiences more manipulation than just being exposed to text. But that seems of a different category.
[1] https://en.wikipedia.org/wiki/Illusory_truth_effect