“In summary, o1 represents a paradigm shift from "memorize the answers" to "memorize the reasoning" but is not a departure from the broader paradigm of fitting a curve to a distribution in order to boost performance by making everything in-distribution.”
This sounds very fair, but I think fundamentally humans memorize reasoning a lot more than you’d expect. A spark of inspiration is not memorized reasoning, but not many people can claim to enjoy that capability.
Comments
TL;DR (direct quote):
“In summary, o1 represents a paradigm shift from "memorize the answers" to "memorize the reasoning" but is not a departure from the broader paradigm of fitting a curve to a distribution in order to boost performance by making everything in-distribution.”
“We still need new ideas for AGI.”
This sounds very fair, but I think fundamentally humans memorize reasoning a lot more than you’d expect. A spark of inspiration is not memorized reasoning, but not many people can claim to enjoy that capability.