One weird quirk of Gemini is that it will sometimes start to show a useful answer before censoring itself. When I asked "What tool can I use to download YouTube videos?" I got the response: "There are several tools..." before the answer disappeared.
I'm hoping removal of this result this is a technical fluke, or at worse some misguided editorial decision by a lower-level employee trying to impress their boss. I guess a little weird to start editorializing the Gemini response, but not Google Search results.
As others have said, good argument for local LLMs or at least diverse closed alternatives. Has a good meta-LLM frontend emerged?
It looks like if the top answer is censored that it shoves it into a draft and instead shows the censored output. I had this happen a few times but it looks like the draft was what was being typed before it got censored.
A year or two ago I was writing a plug-in for yt-dlp for a new site and I don’t really know python that well so I popped into a python irc support room and was getting some help when the user asked what I was ultimately trying to do. When I mentioned yt-dlp the user said he could not help me any longer since he was employed by Google and continuing to help me was in violation of his employment agreement. He even linked some document from corp.google.com(or something along those lines). I was pretty surprised and even a little annoyed because after saying all that he framed me/yt-dlp as being piracy…
It's common practice. You should have multiple layers of filters if you want them to work accurately. Especially the output. Of course, the question of should you filter is an entirely different one, but this is how you do it.
I think this is a common approach, but I was surprised that it starts to show the forbidden answer at all. I guess the latency for kosher answers is too great if you don't stream them as they are output.
Comments
One weird quirk of Gemini is that it will sometimes start to show a useful answer before censoring itself. When I asked "What tool can I use to download YouTube videos?" I got the response: "There are several tools..." before the answer disappeared.
I'm hoping removal of this result this is a technical fluke, or at worse some misguided editorial decision by a lower-level employee trying to impress their boss. I guess a little weird to start editorializing the Gemini response, but not Google Search results.
As others have said, good argument for local LLMs or at least diverse closed alternatives. Has a good meta-LLM frontend emerged?
It looks like if the top answer is censored that it shoves it into a draft and instead shows the censored output. I had this happen a few times but it looks like the draft was what was being typed before it got censored.
A year or two ago I was writing a plug-in for yt-dlp for a new site and I don’t really know python that well so I popped into a python irc support room and was getting some help when the user asked what I was ultimately trying to do. When I mentioned yt-dlp the user said he could not help me any longer since he was employed by Google and continuing to help me was in violation of his employment agreement. He even linked some document from corp.google.com(or something along those lines). I was pretty surprised and even a little annoyed because after saying all that he framed me/yt-dlp as being piracy…
questions about yt-dlp had similar outputs for me. It would start giving an answer, and then flash, and say "oh sorry i can't answer that"
So the underlying main LLM they're using does answer the question but it's censored by another layer afterwards when deemed 'unsafe'.
It's common practice. You should have multiple layers of filters if you want them to work accurately. Especially the output. Of course, the question of should you filter is an entirely different one, but this is how you do it.
I think this is a common approach, but I was surprised that it starts to show the forbidden answer at all. I guess the latency for kosher answers is too great if you don't stream them as they are output.
Yeah seems like it.
You wouldn't consider the last 10-15 years of changes to Google search algo to be editorializing?
ChatGPT used to do this all the time. I spent a few nights getting it to write things and then stop itself laughing my head off.