Comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language ModelparentComments−inciampati4yI've often wondered if a lighter reinforcement learning based model on top of a full text index might do as well or better than these putatively overfit language models. Curious if anyone knows of ongoing or recent work on this approach.
Comments
I've often wondered if a lighter reinforcement learning based model on top of a full text index might do as well or better than these putatively overfit language models. Curious if anyone knows of ongoing or recent work on this approach.