I do wonder though how close the articles that GPT-3 produced were to the articles it had been trained on. For example, the Methodist Church Split article that it produced has a lot of very specific facts about the Methodist Church and about the split, which shows that it had texts about that spefic event in the training set.
It also has a sentence which contains a pretty obvious non-sequitur, but it's easy to miss it or assume that it's a mistake that a human made.
So overall, I'm guessing GPT-3 may actually be pretty decent at re-telling a story with different words, which sometimes is very hard hard to distinguish from a human doing the same thing.
They also don't describe the way they programmatically selected the output, though I am willing to believe that they more or less randomly sampled the output from each model.
Comments
This is a good counterpoint.
I do wonder though how close the articles that GPT-3 produced were to the articles it had been trained on. For example, the Methodist Church Split article that it produced has a lot of very specific facts about the Methodist Church and about the split, which shows that it had texts about that spefic event in the training set.
It also has a sentence which contains a pretty obvious non-sequitur, but it's easy to miss it or assume that it's a mistake that a human made.
So overall, I'm guessing GPT-3 may actually be pretty decent at re-telling a story with different words, which sometimes is very hard hard to distinguish from a human doing the same thing.
They also don't describe the way they programmatically selected the output, though I am willing to believe that they more or less randomly sampled the output from each model.