Humans are really good at subtlety and research. I guess my point is this:
pre-LLM era XZ-style exploit: takes 5 years to attain developer trust, then you merge malicious patches and hope nobody sees them
post-LLM era XZ-style exploit: takes 5 years to attain developer trust, then you merge patches that were written slightly faster and hope nobody sees them
For 99% of people, there isn't a meaningful difference between these situations. I really don't think there are people out there that couldn't write a buffer overflow exploit in 5 years but could groom maintainers for trust in the meantime.
The code-writing and prose bit is the easy part. The social engineering and deception is hard enough for a human, and Turing-assured destruction for LLMs.
Comments
Humans are really good at subtlety and research. I guess my point is this:
For 99% of people, there isn't a meaningful difference between these situations. I really don't think there are people out there that couldn't write a buffer overflow exploit in 5 years but could groom maintainers for trust in the meantime.The code-writing and prose bit is the easy part. The social engineering and deception is hard enough for a human, and Turing-assured destruction for LLMs.