You seem to think AI is more clever than it is. As far as I know, AI tends to produce generic boilerplate code at best and that code becomes extremely hallucinatory beyond non-trivial complexity or when seeking non-commonplace solutions. I don't think anyone's going to be using AI for 1337 h@x any time soon. AI doesn't innovate, it generalizes.
The more pertinent issue is unintentional bugs and low quality in AI generated code, and the effect of knowledge loss as AI generated code becomes more and more common.
Well I guess we just have different standards of "cleverness". As far as I am concerned these technologies already demonstrate a level of subtlety that would allow them to be used in such ways, as is. Nevermind six months from now, or whatever, when it actually reaches a level of refinement where such exploits become trivial (and probably common).
Concerning your second point, as long as there is a human "in the loop" (verifying, testing, etc) that shouldn't really be too much of a concern, should it?
As to the effects of AI-generated content being vacuumed up as training data, that is an interesting question, but also one which I feel is somewhat unresolved at the moment. Is anything really "lost" in the process? Maybe the output becomes more redundant, sure, I just have a hard time believing that that could possibly "break" an LLM. (I have wondered if the mere fact that the output is being fed into the input implies something akin to a "fixed-point iteration" which might somehow become an issue, but again, I honestly don't know!)
Comments
You seem to think AI is more clever than it is. As far as I know, AI tends to produce generic boilerplate code at best and that code becomes extremely hallucinatory beyond non-trivial complexity or when seeking non-commonplace solutions. I don't think anyone's going to be using AI for 1337 h@x any time soon. AI doesn't innovate, it generalizes.
The more pertinent issue is unintentional bugs and low quality in AI generated code, and the effect of knowledge loss as AI generated code becomes more and more common.
Well I guess we just have different standards of "cleverness". As far as I am concerned these technologies already demonstrate a level of subtlety that would allow them to be used in such ways, as is. Nevermind six months from now, or whatever, when it actually reaches a level of refinement where such exploits become trivial (and probably common).
Concerning your second point, as long as there is a human "in the loop" (verifying, testing, etc) that shouldn't really be too much of a concern, should it?
As to the effects of AI-generated content being vacuumed up as training data, that is an interesting question, but also one which I feel is somewhat unresolved at the moment. Is anything really "lost" in the process? Maybe the output becomes more redundant, sure, I just have a hard time believing that that could possibly "break" an LLM. (I have wondered if the mere fact that the output is being fed into the input implies something akin to a "fixed-point iteration" which might somehow become an issue, but again, I honestly don't know!)