I picked the best responses, but everything after the bolded prompt is by GPT-3.
Based on this, I am pretty sure that the order of paragraphs and the general structure (introduction, arguments, conclusion, PS) are entirely the product of the editor, not of GPT-3. I'm assuming that this is at the level paragraphs and not individual sentences, which does leave some pretty good paragraphs.
Another question that I don't know how to answer is how different these paragraphs are to text that is in the training corpus. I would love to see what is the closest bit of text from the whole corpus to each output paragraph.
And finally, human communication and thought is not organized neatly in a uniform level of difficulty from letters to words to sentences to paragraphs to chapters to novels or anything like that, and an AI that can sometimes produce nice-sounding paragraphs is not necessarily any part of the way to actually communicating a single real fact about the world.
I still believe that there is never going to be meaningful NLP without a model/knowledge base about the real physical world. I don't think human written text has enough information to deduce a model of the world from it without assuming some model ahead of time.
I still believe that there is never going to be meaningful NLP without a model/knowledge base about the real physical world.
I think this article is quality enough to constitute meaningful NLP. But, your questions about the amount of human intervention are key. If it takes several hours to a day to produce one of these, then it's not really that meaningful. If one person can produce 100 of these in a day, that's pretty meaningful.
Good catch! The coherent structure was what got me excited. If the structure turns out to be the product of human selection -- and now that you point out the plural on "responses" I think that's likely -- then these results are much more in line with my expectations.
Comments
The article states the following:
Based on this, I am pretty sure that the order of paragraphs and the general structure (introduction, arguments, conclusion, PS) are entirely the product of the editor, not of GPT-3. I'm assuming that this is at the level paragraphs and not individual sentences, which does leave some pretty good paragraphs.
Another question that I don't know how to answer is how different these paragraphs are to text that is in the training corpus. I would love to see what is the closest bit of text from the whole corpus to each output paragraph.
And finally, human communication and thought is not organized neatly in a uniform level of difficulty from letters to words to sentences to paragraphs to chapters to novels or anything like that, and an AI that can sometimes produce nice-sounding paragraphs is not necessarily any part of the way to actually communicating a single real fact about the world.
I still believe that there is never going to be meaningful NLP without a model/knowledge base about the real physical world. I don't think human written text has enough information to deduce a model of the world from it without assuming some model ahead of time.
I think this article is quality enough to constitute meaningful NLP. But, your questions about the amount of human intervention are key. If it takes several hours to a day to produce one of these, then it's not really that meaningful. If one person can produce 100 of these in a day, that's pretty meaningful.
Good catch! The coherent structure was what got me excited. If the structure turns out to be the product of human selection -- and now that you point out the plural on "responses" I think that's likely -- then these results are much more in line with my expectations.