The data is sent to a relay server, which then sends it on to Deepl/Claude.
There are quite a few layers of privacy:
1. Websites on the blacklist (e.g. banking sites), or with certain terms in the URL (e.g. "account"), are blacklisted
2. Websites can be blacklisted by the user
3. Sentences are checked for sensitive language, and filtered for banking info/emails/credentials/etc
4. The words in a sentence must be over a certain proportion (80% iirc) of known English words
5. Once the sentences are sent to the server, the server doesn't log who-sent-what, and the upstream translation services don't know either (because it all comes from the relay).
Translations are cached, though. It's a necessity in order for it to be even remotely economical to do so much translation.
Local LLMs previously weren't good enough, but I've recently done another set of benchmarks (https://nuenki.app/blog/llm_translation_comparison) and llama 3.3 70b is getting there with some languages. Currently I'm in the process of integrating it via Groq, but it could plausibly be done with local LLMs.
Above a certain scale I'll start self hosting, but I'm nowhere near justifying those fixed costs yet.
Would you prefer to run ollama locally, or just know that the translation server is running its own models rather than forwarding onto cloud providers?
I just don't want to send the content of the browsed sites to any third-party service. I am using LanguageTool though, having a local alternative to that will be great too.
Comments
The data is sent to a relay server, which then sends it on to Deepl/Claude.
There are quite a few layers of privacy:
1. Websites on the blacklist (e.g. banking sites), or with certain terms in the URL (e.g. "account"), are blacklisted
2. Websites can be blacklisted by the user
3. Sentences are checked for sensitive language, and filtered for banking info/emails/credentials/etc
4. The words in a sentence must be over a certain proportion (80% iirc) of known English words
5. Once the sentences are sent to the server, the server doesn't log who-sent-what, and the upstream translation services don't know either (because it all comes from the relay).
Translations are cached, though. It's a necessity in order for it to be even remotely economical to do so much translation.
Yeah, I am not worried about the privacy for sending the data to your servers, but for it being forwarded to the cloud LLM models.
Could you add support for local models/Ollama?
Sorry, I missed your reply!
Local LLMs previously weren't good enough, but I've recently done another set of benchmarks (https://nuenki.app/blog/llm_translation_comparison) and llama 3.3 70b is getting there with some languages. Currently I'm in the process of integrating it via Groq, but it could plausibly be done with local LLMs.
Above a certain scale I'll start self hosting, but I'm nowhere near justifying those fixed costs yet.
Would you prefer to run ollama locally, or just know that the translation server is running its own models rather than forwarding onto cloud providers?
I just don't want to send the content of the browsed sites to any third-party service. I am using LanguageTool though, having a local alternative to that will be great too.