Skip to content

Comment on QLoRA: Efficient Finetuning of Quantized LLMsparent

Comments

I'm not sure it could be for this complex quantization scheme. It's not a simple round to nearest quantization. Even if it is possible, it would require someone to develop that functionality. Currently it does load the entire unquantized model into RAM first.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.