Skip to content

Comment on As AI enters the operating room, reports arise of botched surgeries

Comments

Wake up babe, new Therac-25 just dropped.

It’s funny because the Therac-25 failures led the FDA to care about software defects. Before the Therac-25 incidents, software was considered a black box component.

History rhymes doesn’t it?

It’s almost as if we’re an entirely reactive species that can’t think ahead forward on solving problems that aren’t as a result of some massive tragic event

So I wonder then *curiously* what kind of massive tragic event we’re going to have to have before everybody wakes up and realizes superhuman artificial intelligence is something they need to pay attention to politically

I think you’re describing reactive not reactionary.

I'm not totally convinced of this.

https://www.merriam-webster.com/dictionary/reactionary

relating to, marked by, or favoring reaction

It doesn't denote primarily political usage, although that is my interpretation. Here are a few given examples:

The Clippers so far have been reactionary in the playoffs. — Los Angeles Times, 13 June 2021
The demise of the style was spurred by a reactionary resurgence of Classicism. — Regina Cole, Forbes, 1 Jan. 2023
However, the signing had been in the works for weeks and was not a reactionary move in the wake of Sjöberg’s injury. — Daniel Boniface, The Denver Post, 14 Mar. 2017
But in most people, facial bloating and undereye puffiness are usually just reactionary. — Georgia Casey, Allure, 6 Feb. 2026

Agreed, corrected

> It’s almost as if we’re an entirely reactive species that can’t think ahead forward on solving problems that aren’t as a result of some massive tragic event

That's not a characteristic of the species but of specific tendencies within Western civilization. There's certainly people ringing all sorts of alarm bells for digital technology, the climate, food production, etc etc, and they are very often dismissed as "alarmist".

And then there's the motivated reasoning of industrial interests. Think tobacco, asbestos, oil, leaded gasoline, and so on. It's not that we don't know the possible harms, it's just that monetary interests trump the need for health and safety, a.k.a. "health and safety gone mad".

No that’s completely beside the point…

for a computer science degree in the UK to be accredited, the chartered institute mandated an ethics course

therac-25 was in there

and now everyone's shoving as much generative excrement as possibly into everything as quickly as possible

I'm increasingly convinced of the need for professional licensing for software engineering. You get a degree, and sit through that ethics course, and take an exam to get a license, or you don't get to practice. Do unethical things, enjoy major liability.

Software engineers can do just as much damage as civil engineers, automotive engineers, doctors, lawyers, etc...but the industry overall has a disgustingly casual attitude.

I think software engineers in the right place can actually do far more damage than most doctors and lawyers

the scale is simply humongous

just look at what facebook has done to our democracies

Agree, but replace the ethics course with a legal course.

Ethics are just someone else’s biased opinion, legal is put in place by elected officials and has actual consequences.

The law is just someones else encoded ethics for a moment. You don't want your moral to change with the current administration. An ethics course is about thinking about ethics, it is not a mind-altering propaganda action.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.