It's nothing new - even without LLMs you have automated tools that will try stuff to see if your application is vulnerable. You can abuse misconfigured nginx server. To be fair, to your point, LLMs are amazing pattern recognizers = this pattern it has seen in this codebase applies to that codebase so vulnerability is most likely; I'm unsure if they can "innovate" (still, recognizing patterns is enough); this pattern it has seen causes a crash, but I don't know if we're at the point where it can connect two and two together and use a set of unrelated code issues to, for example, exfiltrate credentials