Comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language ModelparentComments−sanxiyn4yWhat's worse, the trend to supersize language models is never justified, either theoretically (ha ha) or empirically in the relevant literatureWhat? That's absurd. Large language models are motivated by empirical scaling law. It is actually better justified than other ML research.Scaling Laws for Neural Language Models: https://arxiv.org/abs/2001.08361
Comments
What? That's absurd. Large language models are motivated by empirical scaling law. It is actually better justified than other ML research.
Scaling Laws for Neural Language Models: https://arxiv.org/abs/2001.08361