Comment on Grokfast: Accelerated Grokking by Amplifying Slow GradientsparentComments−whimsicalism2yi don't think grokking has been demonstrated in a large model yet−HarHarVeryFunny2yThere's a recent paper here where they've seen in a deeper model.https://arxiv.org/abs/2405.19454It's a bit surprising that apparently this hasn't been done before - to see how universal of a phenomenon this is.
Comments
i don't think grokking has been demonstrated in a large model yet
There's a recent paper here where they've seen in a deeper model.
https://arxiv.org/abs/2405.19454
It's a bit surprising that apparently this hasn't been done before - to see how universal of a phenomenon this is.