There are two common ways to spot hallucinations. Talk to it about a topic you know well, or implement what it suggests and get an error or failure.
For the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)
The second still happens, but with much less regularity for me though. It does make mistakes though.
In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.
I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?
This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.
AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given. That's real. Right now. That that is happening anywhere on the planet scares me. That's why I keep asking about AI hallucinating, and it's implications when the stakes are life and death.
The "AIs" being used by these drones have little to do with ChatGPT at the moment. And you should be more worried about "AIs" that do aim at the intended target, anyways.
"hallucination" is a term specific to LLMs like ChatGPT and has virtually nothing to do with visual hallucinations.
It is conceivable that agentic edge LLMs are used for the logic of killer drones but I don't think we're there yet.
Machine learning in general does have classification errors so it may mistakenly see an enemy where there is a pile of rocks(right next to the children's hospital).
To the user of the drone, this represents an error rate and wasted munitions(unless they were planning on bombing the hospital later). So it is something they want to minimize.
Again, I'd be more worried about becoming a "legitimate" target to someone's killer drone than an accidental one.
Comments
There are two common ways to spot hallucinations. Talk to it about a topic you know well, or implement what it suggests and get an error or failure.
For the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)
The second still happens, but with much less regularity for me though. It does make mistakes though.
In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.
I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?
This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.
AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given. That's real. Right now. That that is happening anywhere on the planet scares me. That's why I keep asking about AI hallucinating, and it's implications when the stakes are life and death.
The "AIs" being used by these drones have little to do with ChatGPT at the moment. And you should be more worried about "AIs" that do aim at the intended target, anyways.
I never said anything about ChatGPT.
And that AIs do aim at intended targets with intent to kill while simultaneously being susceptible to hallucinations is exactly what worries me.
"hallucination" is a term specific to LLMs like ChatGPT and has virtually nothing to do with visual hallucinations.
It is conceivable that agentic edge LLMs are used for the logic of killer drones but I don't think we're there yet.
Machine learning in general does have classification errors so it may mistakenly see an enemy where there is a pile of rocks(right next to the children's hospital).
To the user of the drone, this represents an error rate and wasted munitions(unless they were planning on bombing the hospital later). So it is something they want to minimize.
Again, I'd be more worried about becoming a "legitimate" target to someone's killer drone than an accidental one.
It’s a different type of AI. LLMs hallucinate because of their structure — computer vision and algos don’t in the same way