I can't believe everyone is skipping over the most important line:
"The API has zero authorisations checks on cancelling other people's reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you've moved from #4 to #3 already," it messaged back.
The AI systemm didn't hack anything, it lightly touched with a feather duster and the server crumbled.
The AI system probably found swagger documentation of each endpoint, figured that the reservation cancellation API was worth a shot, and then found there was no authentication.
Unauthorized access. 'The door wasn't locked' won't keep you out of jail if you are caught trespassing with intent. Intent makes it interesting, in that Andrew didn't intend computer trespass but the software went and did it anyway on his behalf.
It might not sound like 'hacking' today, but this sort of thing is exactly what it was when the term was invented. Back when you could get free phone calls by whistling into a pay phone or forge emails by telnetting to an SMTP port and setting the Reply-To header to whatever you want.
Comments
I can't believe everyone is skipping over the most important line:
The AI systemm didn't hack anything, it lightly touched with a feather duster and the server crumbled.
The AI system probably found swagger documentation of each endpoint, figured that the reservation cancellation API was worth a shot, and then found there was no authentication.
What is the "hack" here?
Anyone who's thought about it for a single second knows it's immoral to cancel a stranger's appointment without even speaking to them.
If the LLM did not act in line with that incredibly basic understanding, it's clearly misaligned.
-------------
Was it a technically-simple hack?
Sure.
Kevin Mitnick got imprisoned for very simple hacks, usually involving more deceiving of humans than complicated programming prowess.
Nonetheless, the judge and jury found him guilty and sentenced him to jail.
Fundamentally, "hacking" in the "breaking security" sense is about violating trust and common sense social agreements / expectations.
The difficulty involved in so doing is irrelevant.
Unauthorized access. 'The door wasn't locked' won't keep you out of jail if you are caught trespassing with intent. Intent makes it interesting, in that Andrew didn't intend computer trespass but the software went and did it anyway on his behalf.
It might not sound like 'hacking' today, but this sort of thing is exactly what it was when the term was invented. Back when you could get free phone calls by whistling into a pay phone or forge emails by telnetting to an SMTP port and setting the Reply-To header to whatever you want.
I've seen these crud apps. The door is wide open and there is no Swagger or consistent response patterns.
IMO lets name and shame those apps.