I find it useful to keep in mind that LLMs are essentially "just" random generators based on the probability distribution of some language (or more accurately a sample of that distribution in the form of a finite amount of text, a dataset).

You don't expect a random generator to spit out facts, you expect to get random nonsense. But it can be very good at replicating the fitted probability distribution in its output, i.e. generate convincingly coherent language.