Consider the classic ambiguity example from Natural Language Processing: "Time flies like an arrow; fruit flies like a banana".
Markov chains are mostly about memorized local co-occurrence, while transformers are much better at context-sensitive disambiguation and compositional structure.
I do not have deep knowledge about these subjects but this is what I think was meant by "encoding of meaning" (LLMs). I also do not think this capability has to be thought of as "reasoning".
Comments
Working on an "encoding of meaning" sure sounds a lot like reasoning.
Consider the classic ambiguity example from Natural Language Processing: "Time flies like an arrow; fruit flies like a banana".
Markov chains are mostly about memorized local co-occurrence, while transformers are much better at context-sensitive disambiguation and compositional structure.
I do not have deep knowledge about these subjects but this is what I think was meant by "encoding of meaning" (LLMs). I also do not think this capability has to be thought of as "reasoning".