My interest is a little more like flipping that around the other way. Take output from the LLM, get it into structured assertion form, and then use those assertions as part of a query / inference process which pulls in other "known good" (or "ground truth" if you will) assertions from sources like DBPedia or Wikidata. The idea being to either verify the LLM output, or possibly to extend it using inferred conclusions.
I think the way I think about it is somewhat akin to what AWS are doing here, where they talk about using automated reasoning to reduce hallucinations from LLM's:
Comments
My interest is a little more like flipping that around the other way. Take output from the LLM, get it into structured assertion form, and then use those assertions as part of a query / inference process which pulls in other "known good" (or "ground truth" if you will) assertions from sources like DBPedia or Wikidata. The idea being to either verify the LLM output, or possibly to extend it using inferred conclusions.
I think the way I think about it is somewhat akin to what AWS are doing here, where they talk about using automated reasoning to reduce hallucinations from LLM's:
https://aws.amazon.com/blogs/machine-learning/reducing-hallu...
Was looking at prolog and having it shadow the llm activity in order to flag whenever the conversation trips.