your finding that ChatML does not work seems to be mainly this
Also the fact that ChatML has been broken into bits many, many times now- see again the prompt golfing. Also I'm taking OpenAi at their word because they have very strong incentives to pretend to have a solution, and so a public admission that it's currently not solved by the #1 AI company is worth quoting. I'm also just taking their response literally and didn't interpret anything into it.
Indeed, there may be a slight difference in robustness when the inputs are separated by different channels during training and inference. However, my main argument is one from complexity theory- there is no difference here between data & code. Processing the data through a sufficiently advanced model may never be entirely safe. The approach will need to change to constrain these models on well-defined, secure pathways- reducing their utility in the general case. This is very different from SQL injections etc. where we can completely mitigate the issue.
Comments
Also the fact that ChatML has been broken into bits many, many times now- see again the prompt golfing. Also I'm taking OpenAi at their word because they have very strong incentives to pretend to have a solution, and so a public admission that it's currently not solved by the #1 AI company is worth quoting. I'm also just taking their response literally and didn't interpret anything into it.
Indeed, there may be a slight difference in robustness when the inputs are separated by different channels during training and inference. However, my main argument is one from complexity theory- there is no difference here between data & code. Processing the data through a sufficiently advanced model may never be entirely safe. The approach will need to change to constrain these models on well-defined, secure pathways- reducing their utility in the general case. This is very different from SQL injections etc. where we can completely mitigate the issue.