If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Even the 'lowest' humans are usually trying to communicate something when they are telling a story, or teaching you how to do a basic skill, or giving you directions.
GPT-3 can't do any of that. It can pick up clues from the text to produce incredibly realistic sentences that are related to the broad topics of some text, but it's all smoke and mirrors in the end - there is no model of the world getting expressed in communication, it is just mindless aping of similar speech.
And yes, this basic skill that GPT-3 has is enough to replace some human tasks, like inventing plausible sounding stories for a recipe or perhaps even taking news from one site and writing them on another with slight alterations. Perhaps it will even be able to take some facts and weave them into a speech about that topic.
But it is not even close to doing something like real journalism, even at the level of a car mechanic telling you what happened down at the mall.
GPT's text as quoted in the article has a clear thesis stated upfront, later expands on it with examples, summarizes its arguments at the end, and does all this with gusto.
You cap your comment by a non-sequitur that GPT is not going to replace journalism.
IMO the piece of text generated by GPT offers more insight and is wittier than yours.
My comment was not an essay, it was a response to someone else's comment. The GP was explicitly saying that GPT-3 could probably produce many of the news content they read, and I was replying to that.
Please also note that it's not very clear to what extent the text of the article is edited - the non-bold text is written by GPT-3, but I don't think it was produced as a single block of text. Instead different parts (sentences? Paragraphs?) were produced individually, selected by a human from many other responses, and assembled together in the shape we are shown. The train of thought among the paragraphs is most likely entirely human work, not AI work, and only the best sounding paragraphs out of a lot of gibberish were likely selected.
It would also be interesting to see how close those paragraphs are to something in GOT-3's training corpus, in terms of structure if not explicit language.
The authors experimented with 200-word news articles, to see whether 80 human judges could tell the difference between human-generated and GPT-3-generated ones.
It turns out they could not: the human judges correctly identified GPT-3-generated content only 52% of the time, essentially as good as random guessing. (And no, the machine-generated articles were not cherry-picked for the experiment.)
I do wonder though how close the articles that GPT-3 produced were to the articles it had been trained on. For example, the Methodist Church Split article that it produced has a lot of very specific facts about the Methodist Church and about the split, which shows that it had texts about that spefic event in the training set.
It also has a sentence which contains a pretty obvious non-sequitur, but it's easy to miss it or assume that it's a mistake that a human made.
So overall, I'm guessing GPT-3 may actually be pretty decent at re-telling a story with different words, which sometimes is very hard hard to distinguish from a human doing the same thing.
They also don't describe the way they programmatically selected the output, though I am willing to believe that they more or less randomly sampled the output from each model.
> The authors experimented with 200-word news articles, to see whether 80 human judges could tell the difference between human-generated and GPT-3-generated ones.
I disagree with tsimionescu that this is a good counterpoint. The comment you reply to says that "GPT-3 is not saying anything". The figure you refer to shows that human judges could not tell the difference between human-generated and GPT-3 generated text. That's apples and oranges. That some humans weren't able to detect autogenerated text doesn't say anything about whether the autogenerated text said anything.
It may or may not be smoke and mirrors, but there's a clear structure to the essay, as well as a logical structure to the arguments in the essay. For example, "humans think they're intelligent", "humans are wrong about everything", and therefore "intelligence isn't about being right".
It even concludes with reasonably good advice: to pass a Turing test, AIs should say things that are true, and tap into human emotions. To me, this disproves the idea that it's "not saying anything". It's definitely saying something, and that something is both true and not commonly understood by the general public. It is therefore capable of "teaching a basic skill".
I find that very impressive, and I'm surprised that so many others here don't.
As I stated elsewhere, the essay is almost certainly constructed by the human writing the article, out of cherry-picked output by GPT-3 stitched together (I'm guessing at the paragraph level).
Also, again almost certainly, the information about what it takes to pass the Turing test is taken from some text in the corpus it was trained on. What GPT-3 did do is recognize that that is relevant to the topic of AI vs human intelligence, but not much more.
If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Even the 'lowest' humans are usually trying to communicate something when they are telling a story, or teaching you how to do a basic skill, or giving you directions.
you're right; this AI is no substitute for a journalist. however, I do think the "essay" compares favorably with some papers I peer-reviewed for my college writing seminar. sometimes humans really aren't trying to communicate anything; they are just trying to hit the minimum word count and get a passing grade.
And yes, this basic skill that GPT-3 has is enough to replace some human tasks, like inventing plausible sounding stories for a recipe or perhaps even taking news from one site and writing them on another with slight alterations. Perhaps it will even be able to take some facts and weave them into a speech about that topic.
But it is not even close to doing something like real journalism, even at the level of a car mechanic telling you what happened down at the mall.
You're just talking about layers of abstraction. GPT-3 works with chunks of 3 letters. It can now.
Combine chunks to form valid words. Combine words to form syntactic sentences. Combine words to form semantic sentences. Combine sentences to form consistent paragraphs. Combine paragraphs to form trains of thought.
It can't combine trains of thought to produce a consistent point.
But it's already working successfully at 5 or 6 levels of abstraction up. I don't think it's that much harder to get one to two levels higher in abstraction. It doesn't know level 7 is any different from level 6. It just needs the data and compute to start modeling at that level.
In the past you could've convinced me a different architecture is needed to accomplish that. But I also wouldve doubted it could get this far - why would something stop it now?
It just needs to study more long form work and how to reach a conclusion in an essay starting from paragraph 1.
GPT-3 can't do any of that. It can pick up clues from the text to produce incredibly realistic sentences that are related to the broad topics of some text, but it's all smoke and mirrors in the end - there is no model of the world getting expressed in communication, it is just mindless aping of similar speech.
You know this inevitably tempts the cynical question of how different what you describe is from (to put it optimistically) clickbait generators and (to put it still more cynically) much of the content generated today ….
I feel like I can understand it (GPT-3 generated text) better than meme-oriented Reddit callback threads. Which may have more to do with burying meaning more than explicating it.
Comments
If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Even the 'lowest' humans are usually trying to communicate something when they are telling a story, or teaching you how to do a basic skill, or giving you directions.
GPT-3 can't do any of that. It can pick up clues from the text to produce incredibly realistic sentences that are related to the broad topics of some text, but it's all smoke and mirrors in the end - there is no model of the world getting expressed in communication, it is just mindless aping of similar speech.
And yes, this basic skill that GPT-3 has is enough to replace some human tasks, like inventing plausible sounding stories for a recipe or perhaps even taking news from one site and writing them on another with slight alterations. Perhaps it will even be able to take some facts and weave them into a speech about that topic.
But it is not even close to doing something like real journalism, even at the level of a car mechanic telling you what happened down at the mall.
GPT's text as quoted in the article has a clear thesis stated upfront, later expands on it with examples, summarizes its arguments at the end, and does all this with gusto.
You cap your comment by a non-sequitur that GPT is not going to replace journalism.
IMO the piece of text generated by GPT offers more insight and is wittier than yours.
My comment was not an essay, it was a response to someone else's comment. The GP was explicitly saying that GPT-3 could probably produce many of the news content they read, and I was replying to that.
Please also note that it's not very clear to what extent the text of the article is edited - the non-bold text is written by GPT-3, but I don't think it was produced as a single block of text. Instead different parts (sentences? Paragraphs?) were produced individually, selected by a human from many other responses, and assembled together in the shape we are shown. The train of thought among the paragraphs is most likely entirely human work, not AI work, and only the best sounding paragraphs out of a lot of gibberish were likely selected.
It would also be interesting to see how close those paragraphs are to something in GOT-3's training corpus, in terms of structure if not explicit language.
Check out Figure 3.13 in the GPT-3 paper, https://arxiv.org/pdf/2005.14165.pdf.
The authors experimented with 200-word news articles, to see whether 80 human judges could tell the difference between human-generated and GPT-3-generated ones.
It turns out they could not: the human judges correctly identified GPT-3-generated content only 52% of the time, essentially as good as random guessing. (And no, the machine-generated articles were not cherry-picked for the experiment.)
This is a good counterpoint.
I do wonder though how close the articles that GPT-3 produced were to the articles it had been trained on. For example, the Methodist Church Split article that it produced has a lot of very specific facts about the Methodist Church and about the split, which shows that it had texts about that spefic event in the training set.
It also has a sentence which contains a pretty obvious non-sequitur, but it's easy to miss it or assume that it's a mistake that a human made.
So overall, I'm guessing GPT-3 may actually be pretty decent at re-telling a story with different words, which sometimes is very hard hard to distinguish from a human doing the same thing.
They also don't describe the way they programmatically selected the output, though I am willing to believe that they more or less randomly sampled the output from each model.
I disagree with tsimionescu that this is a good counterpoint. The comment you reply to says that "GPT-3 is not saying anything". The figure you refer to shows that human judges could not tell the difference between human-generated and GPT-3 generated text. That's apples and oranges. That some humans weren't able to detect autogenerated text doesn't say anything about whether the autogenerated text said anything.
However, the comment also said this is a way to tell GPT-3 text from human text.
It may or may not be smoke and mirrors, but there's a clear structure to the essay, as well as a logical structure to the arguments in the essay. For example, "humans think they're intelligent", "humans are wrong about everything", and therefore "intelligence isn't about being right".
It even concludes with reasonably good advice: to pass a Turing test, AIs should say things that are true, and tap into human emotions. To me, this disproves the idea that it's "not saying anything". It's definitely saying something, and that something is both true and not commonly understood by the general public. It is therefore capable of "teaching a basic skill".
I find that very impressive, and I'm surprised that so many others here don't.
As I stated elsewhere, the essay is almost certainly constructed by the human writing the article, out of cherry-picked output by GPT-3 stitched together (I'm guessing at the paragraph level).
Also, again almost certainly, the information about what it takes to pass the Turing test is taken from some text in the corpus it was trained on. What GPT-3 did do is recognize that that is relevant to the topic of AI vs human intelligence, but not much more.
you're right; this AI is no substitute for a journalist. however, I do think the "essay" compares favorably with some papers I peer-reviewed for my college writing seminar. sometimes humans really aren't trying to communicate anything; they are just trying to hit the minimum word count and get a passing grade.
Sure, as I said, there are human tasks that may appear creative but aren't.
Also, please note that the 'essay' is assembled by human selection from GPT-3 produced output (whole paragraphs?).
You're just talking about layers of abstraction. GPT-3 works with chunks of 3 letters. It can now.
Combine chunks to form valid words. Combine words to form syntactic sentences. Combine words to form semantic sentences. Combine sentences to form consistent paragraphs. Combine paragraphs to form trains of thought.
It can't combine trains of thought to produce a consistent point.
But it's already working successfully at 5 or 6 levels of abstraction up. I don't think it's that much harder to get one to two levels higher in abstraction. It doesn't know level 7 is any different from level 6. It just needs the data and compute to start modeling at that level.
In the past you could've convinced me a different architecture is needed to accomplish that. But I also wouldve doubted it could get this far - why would something stop it now?
It just needs to study more long form work and how to reach a conclusion in an essay starting from paragraph 1.
Define "semantic" and "trains of thought."
Personally I'd label this human satire, not raw GPT-3 output.
I'll change my mind if I see evidence that proves that's incorrect.
You know this inevitably tempts the cynical question of how different what you describe is from (to put it optimistically) clickbait generators and (to put it still more cynically) much of the content generated today ….
I feel like I can understand it (GPT-3 generated text) better than meme-oriented Reddit callback threads. Which may have more to do with burying meaning more than explicating it.