Skip to content

Comment on So, you want to chunk really fast?

Comments

So, whole english wikipedia in <1 second (~20GB compressed)?

Or is it now a lack of proper pipelining where you first load, then uncompress, then chunk, then write?

Add a nice strong linear model on top like vowpal wabbit and chunk at 100GB/s any language of your choice.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.