Yeah, no kidding. This is what LLMs are not meant for 101 -- it'll get a lot right, a solid amount of the time (spare me the anecdotes), but the idea that someone who doesn't know anything about a topic should take its output for ground truth is exactly the sort of misplaced trust that risks flipping these models from massive technological/productivity/scientific/etcetc leap into existential threat.
Big agree. It feels wrong to dissuade someone from building something, this is likely an exercise in creativity, but I have to say this goes on my "Bad Use of LLMs" list.
Sure, it's possible to "learn" from LLMs in that they might spark some idea that you might not have thought of, but taking the output from LLMs as a source of knowledge is exactly what you shouldn't use an LLM for.
It feels wrong to dissuade someone from building something
I understand this feeling, but it is important to push back against it, most importantly for the developer's own development. Not all ideas are good. I'd easily wager that most ideas are bad or at least not good, at least going from my own thought process. That is why we have places like HN, to share new ideas and get feedback. It is the best thing for the person with the idea (and also society) when their blatantly bad idea is responded to with generous, kind, informative, and direct criticism.
Yeah, it's definitely not at the level where it can be fully trusted yet, but I found GPT (and this) quite helpful for learning about something new where I don't have any background in. That's also why I think that something like showing the source of something might be a good way to improve the trust (although also still not perfect...)
If you don't have any background in a topic, then how do you know it is getting it right? You don't know enough to know if you can trust any given response. Either you have to independently verify what it says with trusted sources, or you just throw your hands up and accept that you're getting taught by the digital equivalent of a once-brilliant professor who did too many hits of LSD.
Valid point, however, I personally believe that the questions generated hold greater significance as they provide direction in terms of what to explore in a particular subject —something that GPT by itself does not provide as it needs to be prompted first—
I don't recall, but I remember posting it here and some other people tried it as well and noticed the same. Anyway, I removed the "book" part of my comment briefly after posting because I thought it detracted from the main point of it.
this particular use of LLM could prove helpful in the sense that it shows you what to look for in a given subject matter. The generated answer can easily be verified.
Comments
Sounds like you reinvented Wikipedia's "Simple English" pages, but in a way that can't be trusted very much.
Yeah, no kidding. This is what LLMs are not meant for 101 -- it'll get a lot right, a solid amount of the time (spare me the anecdotes), but the idea that someone who doesn't know anything about a topic should take its output for ground truth is exactly the sort of misplaced trust that risks flipping these models from massive technological/productivity/scientific/etcetc leap into existential threat.
Big agree. It feels wrong to dissuade someone from building something, this is likely an exercise in creativity, but I have to say this goes on my "Bad Use of LLMs" list.
Sure, it's possible to "learn" from LLMs in that they might spark some idea that you might not have thought of, but taking the output from LLMs as a source of knowledge is exactly what you shouldn't use an LLM for.
I understand this feeling, but it is important to push back against it, most importantly for the developer's own development. Not all ideas are good. I'd easily wager that most ideas are bad or at least not good, at least going from my own thought process. That is why we have places like HN, to share new ideas and get feedback. It is the best thing for the person with the idea (and also society) when their blatantly bad idea is responded to with generous, kind, informative, and direct criticism.
Yeah, it's definitely not at the level where it can be fully trusted yet, but I found GPT (and this) quite helpful for learning about something new where I don't have any background in. That's also why I think that something like showing the source of something might be a good way to improve the trust (although also still not perfect...)
If you don't have any background in a topic, then how do you know it is getting it right? You don't know enough to know if you can trust any given response. Either you have to independently verify what it says with trusted sources, or you just throw your hands up and accept that you're getting taught by the digital equivalent of a once-brilliant professor who did too many hits of LSD.
Valid point, however, I personally believe that the questions generated hold greater significance as they provide direction in terms of what to explore in a particular subject —something that GPT by itself does not provide as it needs to be prompted first—
Another data point, I recently asked ChatGPT for TV show recommendations and got helpful (and not made up) results.
What was the subject and what prompt have you used when asking about the books?
(looks like parent comment has been edited; earlier it mentioned asking GPT for good book recommendations and getting made up results)
I don't recall, but I remember posting it here and some other people tried it as well and noticed the same. Anyway, I removed the "book" part of my comment briefly after posting because I thought it detracted from the main point of it.
Every time I ask an LLM to teach me something I don't know, I am incredibly impressed by the quality of its answer.
But when I ask an LLM to tell me something that I am an expert in, I am usually incredibly disappointed by the bullshit it spews.
To be fair, this is how I often feel reading the news.
this particular use of LLM could prove helpful in the sense that it shows you what to look for in a given subject matter. The generated answer can easily be verified.
Gell-Mann AImnesia?