Comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language ModelparentComments−EvgeniyZh4yNot all weights are born equal, different paradigms allow more parameters while being less parameter-efficient, e.g. https://openreview.net/forum?id=TXqemS7XEH
Comments
Not all weights are born equal, different paradigms allow more parameters while being less parameter-efficient, e.g. https://openreview.net/forum?id=TXqemS7XEH