Comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language ModelparentComments−petters4yIf 16-bit floating-point numbers are used, it can presumably encode all tokens. In theory. It would not be very easy to work with.
Comments
If 16-bit floating-point numbers are used, it can presumably encode all tokens. In theory. It would not be very easy to work with.