This makes it difficult to justify a production deployment:
The core of last_layer is deliberately kept closed-source for several reasons. Foremost among these is the concern over reverse engineering. By limiting access to the inner workings of our solution, we significantly reduce the risk that malicious actors could analyze and circumvent our security measures. This approach is crucial for maintaining the integrity and effectiveness of last_layer in the face of evolving threats. Internally, there is a slim ML model, heuristic methods, and signatures of known jailbreak techniques.
So security by obscurity, to defend llms that are routinely exploited from a position of obscurity. This does not inspire confidence. I'm eagerly awaiting second wave of solutions to this problem that don't take a web app firewall approach where context about what is being defended is absent.
Yeah, I don't like this at all. If I'm going to evaluate a prompt injection protection strategy I need to be able to see how it works.
Otherwise I'm left evaluating it through wasting my time playing whac-a-mole with it, which won't give me the confidence I need because I can't be sure an attacker won't guess a strategy that I didn't think of myself.
This doesn't even include details of the evals they are using! It's impossible to evaluate whether what they've built is effective or not.
I'm also not keen on running a compiled .so file released by a group with no information on even who the authors are.
Comments
This makes it difficult to justify a production deployment:
So security by obscurity, to defend llms that are routinely exploited from a position of obscurity. This does not inspire confidence. I'm eagerly awaiting second wave of solutions to this problem that don't take a web app firewall approach where context about what is being defended is absent.
Yeah, I don't like this at all. If I'm going to evaluate a prompt injection protection strategy I need to be able to see how it works.
Otherwise I'm left evaluating it through wasting my time playing whac-a-mole with it, which won't give me the confidence I need because I can't be sure an attacker won't guess a strategy that I didn't think of myself.
This doesn't even include details of the evals they are using! It's impossible to evaluate whether what they've built is effective or not.
I'm also not keen on running a compiled .so file released by a group with no information on even who the authors are.