I wonder if this is the problem people think it is.
Playing one AI against another is an established technique to developing AI.
Furthermore, content on the internet will always vary from more reliable (well established wiki pages, Reuters) to less reliable (random blog posts, disinformation).
Whether or not an AI generated text doesn't seem to be that important - what's more important is how reliable it is, and how well humans engage with it.
This isn't entirely the right comparison, mostly because "AI" covers a wide span of things.
AI that benefits from the kind of adversarial training that you mention are mostly _planning_ type AIs (think AlphaGo). The problem these AI systems try to solve is that you have some constraints, and as a human "can't be bothered to" work out an optimal solution, so you have the computer do it for you and it does so by starting off with a bad estimate of a solution and "improving" it by trying stuff.
LLMs, on the other hand, are more of a modelling/compression type AI --- (well, more traditionally it wouldn't even be considered AI per-se due to the lack of planning capability). The problem here is to try to represent a huge swath of data in the as efficient a way as possible, thereby forcing it to find and collapse "patterns" by adjusting the representation. Here it's generally not the case that you want to train with adversarial distributions.
An easy thought experiment is: say you take a single case (e.g. a game of Go or a paragraph of text) and massively over-represent it to both models, so much so that that single case eventually covers 99% of all your training instances at the end.
For a "planning" AI this over-representation isn't a huge deal --- once it's learnt all it can from that case, seeing it again is but a waste of time*. It merely makes that particular plan more "clear" but not more "desirable".
However, a modelling/compression type AI will continuously adjust to adapt to the increasing occurrence of that case. It truly "believes" that the more often it sees some pattern, the more important it is, right until it has "forgotten" everything else.
I get that, but not sure if it's a problem in practice. There are two additional consideration's to make:
1) We may be able to model a subset of chatGPT's abilities as adversarial questions. For example, can we write an AI that finds sources and generates questions that a solving AI should be able to figure out. Can we write a test framework such that AI's can challenge themselves to write optimal code for a given solution. Etc.
2) Like I mentioned, if you're scraping the internet you are inherently needing to build some kind of relevance model. E.g. highly up items answers on stack overflow have more weight. In said situation whether or not the content is written by human is largely irrelevant - if you have a reasonable ranking method than highly ranked content is important regardless of source.
Comments
I wonder if this is the problem people think it is.
Playing one AI against another is an established technique to developing AI.
Furthermore, content on the internet will always vary from more reliable (well established wiki pages, Reuters) to less reliable (random blog posts, disinformation).
Whether or not an AI generated text doesn't seem to be that important - what's more important is how reliable it is, and how well humans engage with it.
This isn't entirely the right comparison, mostly because "AI" covers a wide span of things.
AI that benefits from the kind of adversarial training that you mention are mostly _planning_ type AIs (think AlphaGo). The problem these AI systems try to solve is that you have some constraints, and as a human "can't be bothered to" work out an optimal solution, so you have the computer do it for you and it does so by starting off with a bad estimate of a solution and "improving" it by trying stuff.
LLMs, on the other hand, are more of a modelling/compression type AI --- (well, more traditionally it wouldn't even be considered AI per-se due to the lack of planning capability). The problem here is to try to represent a huge swath of data in the as efficient a way as possible, thereby forcing it to find and collapse "patterns" by adjusting the representation. Here it's generally not the case that you want to train with adversarial distributions.
An easy thought experiment is: say you take a single case (e.g. a game of Go or a paragraph of text) and massively over-represent it to both models, so much so that that single case eventually covers 99% of all your training instances at the end. For a "planning" AI this over-representation isn't a huge deal --- once it's learnt all it can from that case, seeing it again is but a waste of time*. It merely makes that particular plan more "clear" but not more "desirable". However, a modelling/compression type AI will continuously adjust to adapt to the increasing occurrence of that case. It truly "believes" that the more often it sees some pattern, the more important it is, right until it has "forgotten" everything else.
*: This is kind of an over-simplification.
I get that, but not sure if it's a problem in practice. There are two additional consideration's to make:
1) We may be able to model a subset of chatGPT's abilities as adversarial questions. For example, can we write an AI that finds sources and generates questions that a solving AI should be able to figure out. Can we write a test framework such that AI's can challenge themselves to write optimal code for a given solution. Etc.
2) Like I mentioned, if you're scraping the internet you are inherently needing to build some kind of relevance model. E.g. highly up items answers on stack overflow have more weight. In said situation whether or not the content is written by human is largely irrelevant - if you have a reasonable ranking method than highly ranked content is important regardless of source.