Comment on LLM.int8(): 8-Bit Matrix Multiplication for Transformers at ScaleComments−ofirpressOP4yCool new efficient inference method that saves 2x memory and does not degrade performance for large language models!More from the author about this at: https://twitter.com/Tim_Dettmers/status/1559892888326049792
Comments
Cool new efficient inference method that saves 2x memory and does not degrade performance for large language models!
More from the author about this at: https://twitter.com/Tim_Dettmers/status/1559892888326049792