Prompt injection is the security flaw that exists because doing that - treating instructions and data as separate things in the context in the LLM - is WAY harder than you might expect.
Then we should improve the tooling around this to make it way easier, rather than hoping security by obscurity will work this time.
AI labs around the world have been trying to solve this problem - reliable separation of instructions from data for LLMs - for a year and a half at this point. It's hard.
Comments
Then we should improve the tooling around this to make it way easier, rather than hoping security by obscurity will work this time.
AI labs around the world have been trying to solve this problem - reliable separation of instructions from data for LLMs - for a year and a half at this point. It's hard.