Skip to content

Ask HN: How to store KV cache efficiently on disk?

1 pointban-lan-gendiscuss
On HN

If I have a lot of precomputed KV cache values (basically just tensors), how do I store them efficiently on disk? Ideally, I would want to lookup their values by prefix quickly as well.

Comments

No comments yet.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.