I ran into this building data pipelines for LLMs. Kept feeding structured JSON with numbers and the model would do bizarre arithmetic on them.
The moment I shifted to computing the analysis upstream and describing results in plain english, the outputs were much more coherent.
The model doesn't know what numbers/spreadsheets of data mean inherently, as in... the LLM inherently does not compute a math equation/formula within itself. Rather, an LLM would calls/creates calculation code on the side to then read off the results. And still with this side-work, the code/calculator created would give contextual wordings to the numbers; highest, average, x% growth, consolidating at y% rate, etc.
Comments
I ran into this building data pipelines for LLMs. Kept feeding structured JSON with numbers and the model would do bizarre arithmetic on them.
The moment I shifted to computing the analysis upstream and describing results in plain english, the outputs were much more coherent.
The model doesn't know what numbers/spreadsheets of data mean inherently, as in... the LLM inherently does not compute a math equation/formula within itself. Rather, an LLM would calls/creates calculation code on the side to then read off the results. And still with this side-work, the code/calculator created would give contextual wordings to the numbers; highest, average, x% growth, consolidating at y% rate, etc.