Skip to content

Comment on GigaToken: ~1000x faster Language model tokenizationparent

Comments

Just to clarify, latency is one form of performance, and a separate thing to optimize from total resource usage in more classic "performance critical" situations. That performance might be energy, space, or other dimensions besides latency. It might also be something like reliability, accuracy, precision, or even the very human factors like simplicity, modifiability, and visibility.

Heck, even latency alone you can just reduce the standard deviation and get smoother flows. Little's law is a great callout here too, one of my favorite computer science principles.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.