"predicting the next token well means that you understand the underlying reality that led to the creation of that token"

People on the AI-hype side of things tend to believe this, but I really fundamentally don't.

It's become a philosophical debate at this point (what does it mean to "understand" something, etc.)