It honestly worries me when I see people paint the whole "everything is a hallucination" angle as uniquely applying LLMs...

If you're going to be that loose with your definition of hallucination, I'd really hope you apply it to yourself. That's a level of introspection that you need to avoid letting biases that all of us have deep down end up affecting your higher order function.

We humans rely on a ball of biases, interpolations and extrapolations to function, LLMs don't have a monopoly on that.