Failure to learn seems almost synonymous with overfitting. It’s the failure to distinguish noise from signal due to lack of data (or just lack of imagination and ability to quickly see the general pattern, or even that there might be one).
That would indeed be a nice use of LLMs, although for mathematics I wouldn’t feel I could trust them not to warp the meaning badly. You’d probably have to check each generation carefully by hand.
Comments
This isn't exactly what you are saying, but I have noticed that students basically 'overfit' the training data on multiple-choice tests.
A nice use of LLMs might be to rewrite these questions on the fly so they test the same understanding while making it difficult to memorize answers.
Failure to learn seems almost synonymous with overfitting. It’s the failure to distinguish noise from signal due to lack of data (or just lack of imagination and ability to quickly see the general pattern, or even that there might be one).
That would indeed be a nice use of LLMs, although for mathematics I wouldn’t feel I could trust them not to warp the meaning badly. You’d probably have to check each generation carefully by hand.