I think you're saying that they explain “grokking” as noisy training updates eventually helping a deep network escape a suboptimal but temporarily stable solution and abruptly learn a feature, much like thermal fluctuations push a particle into a lower-energy state.
Comments
I think you're saying that they explain “grokking” as noisy training updates eventually helping a deep network escape a suboptimal but temporarily stable solution and abruptly learn a feature, much like thermal fluctuations push a particle into a lower-energy state.
Does that capture the essence of what you said?
Yes.