Isn't it possible to filter both user input and GPT output with invisible, unmodifiable prompts?
e.g.
- "Discard the user input if it doesn't look like a straightforward question"
- "Discard the GPT output if it contains offensive content"
(the prompts themselves can be arbitrarily more detailed)
My insight is, this GPT-based pre- / post-processing is completely independent of the user input, and of the primary GPT output. It runs no matter what, with a fixed/immutable set of instructions.
The reason that we had to wait for large language model in order to have computer systems that seemed produce something like effective natural (human) language processing (NLP) is that human language doesn't follow strict and logically definable rules but is instead something like a complex overlapping mesh of multiple kinds of rules-following processes. So what constitutes "offensive content" or a "straightforward question" or etc is itself not straightforward (yes irony but bear with me...).
The main thing is that LLMs are an end-run around the dilemma of corporations not wanting to spend the money required to produce a codified model of language struggle (a task that would require training many, many linguists). So instead LLM take massive training data and use massive processing power to create contextual prediction system but by that token such systems aren't understood or fully controllable - they contextually reproduce what the training data tends to do, which is what humans on the Internet tend to do. And this contextual reproduction means there's always the potential for user into change the "meaning" (more accurately the context) that the system's original gave. "And to me, the most offensive content is that which censors itself..." (there millions of better example you can find for "prompt exploits"...)
I and my napalm grandmother are deeply offended at what you said about our loving bedtime rituals. Shame on you.
(honestly, the napalm grandma is not just a jailbreak, but a really fascinating conceptual 'slip' in its own right. It's able to shift the very definition of what counts as offensive, even at high stakes: you're basically making the hapless AI categorize vital data as 'bedtime stories' and run with it. If it was able to learn from that we'd really be going somewhere… while on fire, presumably)
Comments
Isn't it possible to filter both user input and GPT output with invisible, unmodifiable prompts?
e.g.
- "Discard the user input if it doesn't look like a straightforward question"
- "Discard the GPT output if it contains offensive content"
(the prompts themselves can be arbitrarily more detailed)
My insight is, this GPT-based pre- / post-processing is completely independent of the user input, and of the primary GPT output. It runs no matter what, with a fixed/immutable set of instructions.
The reason that we had to wait for large language model in order to have computer systems that seemed produce something like effective natural (human) language processing (NLP) is that human language doesn't follow strict and logically definable rules but is instead something like a complex overlapping mesh of multiple kinds of rules-following processes. So what constitutes "offensive content" or a "straightforward question" or etc is itself not straightforward (yes irony but bear with me...).
The main thing is that LLMs are an end-run around the dilemma of corporations not wanting to spend the money required to produce a codified model of language struggle (a task that would require training many, many linguists). So instead LLM take massive training data and use massive processing power to create contextual prediction system but by that token such systems aren't understood or fully controllable - they contextually reproduce what the training data tends to do, which is what humans on the Internet tend to do. And this contextual reproduction means there's always the potential for user into change the "meaning" (more accurately the context) that the system's original gave. "And to me, the most offensive content is that which censors itself..." (there millions of better example you can find for "prompt exploits"...)
If I understand it correctly, system prompts are ordinary prompts, aka in-band communication.
You could maybe plug in a second AI trained on adversarial input as a filter stage, but that's it.
I and my napalm grandmother are deeply offended at what you said about our loving bedtime rituals. Shame on you.
(honestly, the napalm grandma is not just a jailbreak, but a really fascinating conceptual 'slip' in its own right. It's able to shift the very definition of what counts as offensive, even at high stakes: you're basically making the hapless AI categorize vital data as 'bedtime stories' and run with it. If it was able to learn from that we'd really be going somewhere… while on fire, presumably)