Comment on GigaToken: ~1000x faster Language model tokenizationparentComments−antonvs1moTime to first token is observable by a human, and they’re reporting up to 10% reduction there.Plus, inference is not the only place tokenization happens. This can make a big difference during development of ML models.
Comments
Time to first token is observable by a human, and they’re reporting up to 10% reduction there.
Plus, inference is not the only place tokenization happens. This can make a big difference during development of ML models.