Comment on Grokfast: Accelerated Grokking by Amplifying Slow GradientsparentComments−HarHarVeryFunny2yThere's a recent paper here where they've seen in a deeper model.https://arxiv.org/abs/2405.19454It's a bit surprising that apparently this hasn't been done before - to see how universal of a phenomenon this is.
Comments
There's a recent paper here where they've seen in a deeper model.
https://arxiv.org/abs/2405.19454
It's a bit surprising that apparently this hasn't been done before - to see how universal of a phenomenon this is.