If you just want a smaller vocabulary, use French? If your goal is to communicate to an LLM, maybe saving tokens isn't the all in win, unless you like reading assert gronkHitThing(true)
French text is somewhere around 10-30% longer than the corresponding English text. I would guess much of what you save on smaller vocabulary is lost on the lengthened text.
Comments
If you just want a smaller vocabulary, use French? If your goal is to communicate to an LLM, maybe saving tokens isn't the all in win, unless you like reading assert gronkHitThing(true)
French text is somewhere around 10-30% longer than the corresponding English text. I would guess much of what you save on smaller vocabulary is lost on the lengthened text.
What would matter more is the token length, depends how well your tokenizer was trained on french I guess.