Skip to content

Comment on AI companies charge you 60% more based on your language, BPE tokensparent

Comments

Wait until you hear that most models tend to perform worse for non-English languages.

Do you know if that's true of non-English models?

As I said elsewhere, Deepseek injects Chinese characters into responses. Anecdotally, that seems to happen when the context gets longer. That suggests that they're primarily trained in Chinese and I would expect them to use fewer tokens for Chinese than English.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.