I’ve tried to identify parts of the prompt in the output, hopefully recognizing someone trying to dump the prompt, but that’s pretty tough to do. I’m sure even fine tuned models are susceptible to spilling proprietary training data
Our paper argues that this might have significant security implications beyond spilling the original prompt or training data when models are integrated with other applications (like search).
We showcase completely new methods to:
- deliver/inject adversarial prompts
- remotely control LLMs
- deliver hidden multi-stage payloads
- spreading payloads/injections to other application-integrated LLMs
- manipulating data
- exfiltrating arbitrary user data with only search capabilities
- target code completion engines
- target automated systems
So, recognizing parts of the prompt or fine tuning may not be sufficient mitigations.
Comments
I’ve tried to identify parts of the prompt in the output, hopefully recognizing someone trying to dump the prompt, but that’s pretty tough to do. I’m sure even fine tuned models are susceptible to spilling proprietary training data
Our paper argues that this might have significant security implications beyond spilling the original prompt or training data when models are integrated with other applications (like search). We showcase completely new methods to:
- deliver/inject adversarial prompts
- remotely control LLMs
- deliver hidden multi-stage payloads
- spreading payloads/injections to other application-integrated LLMs
- manipulating data
- exfiltrating arbitrary user data with only search capabilities
- target code completion engines
- target automated systems
So, recognizing parts of the prompt or fine tuning may not be sufficient mitigations.