Our paper argues that this might have significant security implications beyond spilling the original prompt or training data when models are integrated with other applications (like search).
We showcase completely new methods to:
- deliver/inject adversarial prompts
- remotely control LLMs
- deliver hidden multi-stage payloads
- spreading payloads/injections to other application-integrated LLMs
- manipulating data
- exfiltrating arbitrary user data with only search capabilities
- target code completion engines
- target automated systems
So, recognizing parts of the prompt or fine tuning may not be sufficient mitigations.
Comments
Our paper argues that this might have significant security implications beyond spilling the original prompt or training data when models are integrated with other applications (like search). We showcase completely new methods to:
- deliver/inject adversarial prompts
- remotely control LLMs
- deliver hidden multi-stage payloads
- spreading payloads/injections to other application-integrated LLMs
- manipulating data
- exfiltrating arbitrary user data with only search capabilities
- target code completion engines
- target automated systems
So, recognizing parts of the prompt or fine tuning may not be sufficient mitigations.