I tested on the dickens dataset (Charles Dickens novels). This algorithm works well on English text. I do not expect it to be any better than zstd at binary data. I haven't tried code.
It does used zstd. My algorithm is a preprocessing step for zstd.
Comments
I tested on the dickens dataset (Charles Dickens novels). This algorithm works well on English text. I do not expect it to be any better than zstd at binary data. I haven't tried code.
It does used zstd. My algorithm is a preprocessing step for zstd.
Commit that data + test code :)
Add the results to the readme ::))