It would absolutely get through FDA approvals. You don't know what you're talking about.
The FDA would require extensive efficacy trials and an inventory of side-effects, the same as other devices. The FDA isn't doing computer engineering, it's measuring outcomes and risks.
they won't trust it end-to-end, but they will allow AI to generate code and have humans perform final review, at least at first. That requires way fewer developers.
In my view the difference comes down to accountability. In all of the heavily-regulated environments I’ve worked in, humans are responsible for compliance in systems.
If an LLM generates vulnerable code that evades detection and makes it into production, and that vulnerability is exploited, who is responsible? Presumably, the humans who were tasked with reviewing and approving the code.
As the famous quote states, it is easier to write code than to read it. An LLM would be great for improving developer tooling to generate boilerplate code, etc., but I see tremendous risk for any firms attempting to let an LLM design and build any substantial non-trivial pieces of system code. And the effort required to review the code is at least on the same order of magnitude as it would be to write the code (beyond the trivial “make me an HTTP controller or HTML form for these operations” and the like)
Comments
No way any regulated industry is going to just trust some AI to write the code.
Imagine trying to get a medical device through FDA approval with firmware written by a bot.
It would absolutely get through FDA approvals. You don't know what you're talking about.
The FDA would require extensive efficacy trials and an inventory of side-effects, the same as other devices. The FDA isn't doing computer engineering, it's measuring outcomes and risks.
Or imagine trying to provision a data layer that supports a distributed backend. Good luck basing your cloud costs on a word calculator.
they won't trust it end-to-end, but they will allow AI to generate code and have humans perform final review, at least at first. That requires way fewer developers.
How is it different from today?
In my view the difference comes down to accountability. In all of the heavily-regulated environments I’ve worked in, humans are responsible for compliance in systems.
If an LLM generates vulnerable code that evades detection and makes it into production, and that vulnerability is exploited, who is responsible? Presumably, the humans who were tasked with reviewing and approving the code.
As the famous quote states, it is easier to write code than to read it. An LLM would be great for improving developer tooling to generate boilerplate code, etc., but I see tremendous risk for any firms attempting to let an LLM design and build any substantial non-trivial pieces of system code. And the effort required to review the code is at least on the same order of magnitude as it would be to write the code (beyond the trivial “make me an HTTP controller or HTML form for these operations” and the like)