On-the-fly snapshot compression for elastic inference at scaleblog.doubleword.ai 6somnial1mo1 comment
Pushing memory bound CUDA kernels past the speed of light with data compressionfergusfinn.com 2somnial3modiscuss
Speculative KV coding: ~4× losslessly compressed KV cache using a small modelfergusfinn.com 2somnial4modiscuss