Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapsesquesma.com 15 pointsstared16 days ago2 commentsSaveHideCopy link On HNComments−Gerard22Aug16dYeah, there is latest and greatest trend: 0-bit quant. Absolutely outstanding performance, can produce as many tokens per second as you terminal may sustain with zero load on unified memory / GPU. Real winner!−kamranjon16dHmm, makes the 2 bit quants actually seem pretty reasonable…
Comments
Yeah, there is latest and greatest trend: 0-bit quant. Absolutely outstanding performance, can produce as many tokens per second as you terminal may sustain with zero load on unified memory / GPU. Real winner!
Hmm, makes the 2 bit quants actually seem pretty reasonable…