One surprising (to me at least) benefit of hooking up an LLM to your docs is that it is actually a really useful way to find gaps in your docs. For example, when an LLM cannot answer a user question, there's a good chance it's because the answer is not documented anywhere.
Confabulation should not be stable. Ie. you can generate answer let's say 3 times with different seed / non-zero temp and if it arrives at different answers you can categorize it as confabulation - which likely means, again, that the answer is not present and humans will likely also have different interpretations.
Comments
Exactly! It's very much garbage-in-garbage-out.
One surprising (to me at least) benefit of hooking up an LLM to your docs is that it is actually a really useful way to find gaps in your docs. For example, when an LLM cannot answer a user question, there's a good chance it's because the answer is not documented anywhere.
What about when it answers with confabulation?
Confabulation should not be stable. Ie. you can generate answer let's say 3 times with different seed / non-zero temp and if it arrives at different answers you can categorize it as confabulation - which likely means, again, that the answer is not present and humans will likely also have different interpretations.
I guess then it becomes a threat to Google's business model.