Yes yes yes! I've been anticipating this for so long.
However I'm a bit skeptical about doing quantization after training, in my experience you have to do quantization-aware training for there not be a large performance decrease. I guess it works though otherwise they wouldn't have released it?
https://arxiv.org/abs/1906.04721 is an example paper where they perform data-free quantization after training (using batchnorm parameters to get information about the data distribution) without large performance drops.
Comments
Yes yes yes! I've been anticipating this for so long.
However I'm a bit skeptical about doing quantization after training, in my experience you have to do quantization-aware training for there not be a large performance decrease. I guess it works though otherwise they wouldn't have released it?
According to their docs[1], three different ways of quantizing your model are supported - one of which is quantization-aware training (3).
[1] - https://pytorch.org/docs/master/quantization.html#quantizati...
https://arxiv.org/abs/1906.04721 is an example paper where they perform data-free quantization after training (using batchnorm parameters to get information about the data distribution) without large performance drops.
i hate microsoft's way of getting things done! stop reinventing the wheel !