Comment on So, you want to chunk really fast?Comments−vjerancrnjak8moSo, whole english wikipedia in <1 second (~20GB compressed)?Or is it now a lack of proper pipelining where you first load, then uncompress, then chunk, then write?Add a nice strong linear model on top like vowpal wabbit and chunk at 100GB/s any language of your choice.
Comments
So, whole english wikipedia in <1 second (~20GB compressed)?
Or is it now a lack of proper pipelining where you first load, then uncompress, then chunk, then write?
Add a nice strong linear model on top like vowpal wabbit and chunk at 100GB/s any language of your choice.