Comment on GigaToken: ~1000x faster Language model tokenizationparentComments−dingdingdang1moHave you done preliminary numbers on replacing tokenizer on, say, llama-server?−marcelroed1moAdded numbers here: https://news.ycombinator.com/item?id=49015014−marcelroed1moRunning the numbers now
Comments
Have you done preliminary numbers on replacing tokenizer on, say, llama-server?
Added numbers here: https://news.ycombinator.com/item?id=49015014
Running the numbers now