Thanks for the demo video; very cool. How do you prevent prompt injection attacks?
E.g., if you create a user account for the LLM models to run in, how to do prevent an attack where the LLMs can be leveraged to execute privilege escalation to get as close to admin privs as possible? Or, how to ensure PII/PHI standards in a functionality such as this?
The LLM only communicates with the shell so other users shouldn’t be able to leverage it. Maybe if the LLM reads a file that a lower privilege user has access to that’d be a route for injection into the context at least. There’s also a ‘command protection’ system we’re trying out that uses a separate secondary LLM to approve the commands of the primary.
Comments
Thanks for the demo video; very cool. How do you prevent prompt injection attacks?
E.g., if you create a user account for the LLM models to run in, how to do prevent an attack where the LLMs can be leveraged to execute privilege escalation to get as close to admin privs as possible? Or, how to ensure PII/PHI standards in a functionality such as this?
The LLM only communicates with the shell so other users shouldn’t be able to leverage it. Maybe if the LLM reads a file that a lower privilege user has access to that’d be a route for injection into the context at least. There’s also a ‘command protection’ system we’re trying out that uses a separate secondary LLM to approve the commands of the primary.