The problem hasn't been solved, and, by the very nature of what LLMs are, can't be. However, it has gotten a lot better, as models have got much larger and pretraining more sophisticated.
Part of the difficulty--not in solving, but in discussing--is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.
I prefer to say that it's all hallucinations so the time... we just happen to like some portion of them.
_______
In fact [The Bursar] was incurably insane and hallucinated more or less continuously, but by a remarkable stroke of lateral thinking his fellow wizards had reasoned that, in that case, the whole business could be sorted out if only they could find a formula that caused him to hallucinate that he was completely sane.
This had worked well. There had been a few false starts. For several hours, at one point, he had hallucinated that he was a bookcase. But now he was permanently hallucinating that he was a bursar, and that almost made up for the small side-effect that also led him to hallucinate that he could fly.
Comments
The problem hasn't been solved, and, by the very nature of what LLMs are, can't be. However, it has gotten a lot better, as models have got much larger and pretraining more sophisticated.
Part of the difficulty--not in solving, but in discussing--is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.
I prefer to say that it's all hallucinations so the time... we just happen to like some portion of them.
_______
—The Truth (2000) by Terry Pratchett