Show HN: Evaluating Local LLMs as language translators for my app
lector.devThis is my first attempt at running an eval of this nature so would love some methodology feedback.
I can't guarantee the sources weren't already in the model's inputs without getting novel translations from native speakers, but from my experience using the top models, they feel very accurate. Even encountering somewhat obscure texts from a relatively small language the translations generally beat Google Translate for proper idiomatic meaning.
Comments
Interesting study! First time I've really looked at this sort of study so probably not the best to comment on methodology but the overall premise of using LLMs for language conversion to increase an apps TAM is appealing
Thank you! Running phrase translations through an LLM really helps with processing idiomatic phrasing, and also enables quite good output practice. Spoken output practice would be a game changer.
There's definitely a lot of potential in this space especially in generating learner content in languages where there's sparse content online.