The companies didn't arbitrarily choose to bill by tokens. The cost to serve the models scales linearly with tokens which makes it a reasonable pricing strategy. The reality is that you are charged more because it was more expensive to handle the request.
I guess token length is indirectly determined by language frequency in the training set, and it would be possible to train a model on machine translated training data only to combat that (or maybe to force tokenization to overrepresent languages other than English?), but there’s no way that would be economical, and inference would just be accordingly more expensive to recoup that effort.
Comments
The companies didn't arbitrarily choose to bill by tokens. The cost to serve the models scales linearly with tokens which makes it a reasonable pricing strategy. The reality is that you are charged more because it was more expensive to handle the request.
I guess token length is indirectly determined by language frequency in the training set, and it would be possible to train a model on machine translated training data only to combat that (or maybe to force tokenization to overrepresent languages other than English?), but there’s no way that would be economical, and inference would just be accordingly more expensive to recoup that effort.