Skip to content

Comment on Grokfast: Accelerated Grokking by Amplifying Slow Gradientsparent

Comments

Several reasons - computational effort, effort it takes to reproduce results on complex datasets, academic publishing model.

Academic research is often times about making incremental steps and limiting uncertainty.

Making something work for MNIST is already so much work, researchers don’t have the time, money, and energy to run experiments for 10 datasets.

Complex datasets are much harder to get a proper model trained on due to increased complexity - larger images, tasks, classes, etc.

Also, as soon as you run your experiments on more datasets, you create an opportunity for reviewers to take you down - “why didn’t you test it on this other dataset?”

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.