I’ve been in a small group at an academic institution. With our meager resources we trained larger models than this on many different vision problems. I personally train LLMs on OpenWebText than this using a few 4090s (not work related). Is that too much for a small group?
MNIST is solvable using two pixels. It shouldn’t be one of two benchmarks in a paper, again just in my opinion. It’s useful for debugging only.
I thought so at first, but the repo's[0] owner and the first name listed in the article has Seoul National University on their Github profile.
Far away from a small academic institution.
Oh hm, so they are. I thought they were binary because they used a digital pen to create them, IIRC, and logistic regression is always the baseline; but checking, they technically are grayscale and people don't always binarize them. So I guess information-theoretically, if they are 0-255 valued, then 2 pixels could potentially let you classify pretty well if sufficiently pathological.
Comments
I’ve been in a small group at an academic institution. With our meager resources we trained larger models than this on many different vision problems. I personally train LLMs on OpenWebText than this using a few 4090s (not work related). Is that too much for a small group?
MNIST is solvable using two pixels. It shouldn’t be one of two benchmarks in a paper, again just in my opinion. It’s useful for debugging only.
Again, a small academic institution may not have the experience or know-how to know these things.
I thought so at first, but the repo's[0] owner and the first name listed in the article has Seoul National University on their Github profile. Far away from a small academic institution.
[0]: https://github.com/ironjr/grokfast
It's a free world. Nothing stops you from applying their findings to bigger datasets. It would be a valuable contribution.
How can MNIST be solved using just two binary pixels when there's 10 classes, 0-9?
i'm also curious but my understanding was MNIST pixels are not binary due to some postprocessing artifacts
Oh hm, so they are. I thought they were binary because they used a digital pen to create them, IIRC, and logistic regression is always the baseline; but checking, they technically are grayscale and people don't always binarize them. So I guess information-theoretically, if they are 0-255 valued, then 2 pixels could potentially let you classify pretty well if sufficiently pathological.
really? do you have any details?
agree it has no business being in a modern paper