I think as an existing expert in my field (robot devops) what I feel most strongly is not that I'm losing my skills but rather than I'm losing my nerve.
Like, problems that I would previously dig into on my own, poking at this and that log file to try to understand what happened to get a system into a particular state, now claude is mediating most of those kinds of interactions and it is the one doing the first pass surfacing of "okay I discovered X, Y, and Z things that are hinky, and I'm not totally sure yet what this all means, but let's look together."
Nowadays I feel almost naked looking at a terminal where I'm typing each character myself. A lot of the old instincts around tab completion and grepping through the --help output of every tool, all that stuff has atrophied somewhat. Maybe that is genuine skill decline?
I think this is a really important idea. Good problem solvers, in general, are people who are prepared to go in and poke around, to see what happens when you disable this or add that.
The only way I've found to make juniors develop that instinct is to make them do it until it comes naturally--to continually demand they verify their assumptions, test out theories while debugging. If you don't practice being comfortable with the unfamiliar, you lose the knack.
It's fine, of course, to lose the knack to AI, as long as you'll never need to solve a problem the AI can't (or can't access). I don't think that's a good assumption for everyone to make.
Good problem solvers, in general, are people who are prepared to go in and poke around, to see what happens when you disable this or add that.
you have to do all this when dealing with AI too. The syntax is a conversation vs a strict grammar but you still have to do all the exploration, verification, and evaluation you always had to do as a developer. You have to know how to explain what you want and know when you have it, that's not always an easy skill to learn though.
The truly wild situation for me is when I have something self contained enough that I can let the bot manage the entire loop unsupervised. Like, here's the patches from the old version, here's the new version, here's the test suite that's got to pass. I'm going to bed now, see you in the morning.
And in the morning, I have patches for the new version that pass the test suite.
More people that are not devs, are using AI to create things, and are skilling up to a mediocre level?
What skills are being developed by non-devs prompting AI to churn out code they don't understand? Note that I'm not debating if the code is good or works or whatever, I'm asking if any real skills are being developed merely by prompting AI towards some goal.
my sister and two of her friends put together a small company. She asked me to work on a mobile app for them, like a small niche LMS. I was making progress but fell behind, i have a demanding job/family and she doesnt. She asked if she could try to take it over, i told her about claudecode and then gave her some magic words to get started with. Basically, some framework names and some other technologies involved and some prompt guidance. She completely took off with it and came up with a flow where she chats with ChatGPT and it tells her what to tell ClaudeCode, she copy/pastes into claudecode does it's thing.
How could it possibly work? Except she got through setting up her own Apple dev account, all the certificates, subscription entitlement, revenuecat integration, TestFlight, all the paperwork that goes along with a listing and all of that in about 3 weeks. She submitted her app to the AppStore for review last friday.
She didn't learn any coding but she sure as hell produced a pretty good mobile app with a subscription payment system all on her own with the help of ChatGPT and ClaudeCode. She's also producing different tools that run on her website which is hosted on squarespace. I have no idea how she's doing it but she has all these little questionairre and decision tree pages backed by google sheets. She's the second non-coder i've seen really blossom with AI and just run with it building stuff she's always wanted but never knew how.
If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero.
Its just what they are learning, is not as much as they think, and really random, not-structured. Like reading just a few random chapter of a book across multiple classes. They get a smattering of ad-hoc tidbits of knowledge.
I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.
If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero.
I think the knowledge being absorbed is 'how to use AI to make apps', not necessarily anything about the code itself. I've seen lots of people who are very effective with AI suddenly poleaxed in interviews without it. Going from 'able to explain exactly what needs to happen' to 'staring at a blank page with no ideas'.
I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.
It absolutely is. It removes a lot of friction from pathways that kept previous knowledge firm.
This isn't unique to AI. When calculators came around, people had a new tool that decreased the need to do arithmetic in your head. This has lead to a decline in the percentage of people who knew arithmetic, but can no longer do it adequately without the external tool. Another example would be assembly skills declining amongst software engineers after compiled languages become dominant.
What I would argue is unique to AI is two-fold. The first being that the scale of things being automated is enormous. Keeping with just software, AI isn't only a tool that decreases the need to write code, it's also a tool that decreases the need to do your own research, debugging, system design, version control, etc. The list of tasks a regular engineer is responsible for that have not been consumed by AI is small. To make matters more frightening, this is just a single industry. The same applies to multiple other knowledge heavy fields (e.g. mathematics).
The second thing I'd argue is unique about AI is the addictive nature of it. These LLMs are deliberately coerced into being sycophants that blow smoke up your ass unless you deliberately tell the model not too (and it may still do it anyways!). A calculator didn't compliment your insight for asking it to calculate the square root of -1.
Furthermore, many people have talked about the resemblance of LLMs to slot machines--there's a feeling that anything could be possible if you only prompt the model correctly. Your first prompt is wildly off the mark, but your next prompt is better though still not quite right, so you continue forward. You keep iterating your instructions, your word choices, your tone, all in search of the desired response. In this way, an LLM is a slot machine, your prompt the lever you pull, and the output is the jackpot.
All this taken together, we have a machine that decreases the need to engage with the critical faculties of your own brain, which has been trained to keep your attention through flattery, and whose probabilistic nature preys on our love for gambling.
I've got a friend whose firm has started using 'cognitive surrender' to describe the habit of delegating thinking to the LLM in the way that we used to skim code reviews and say 'LGTM' while missing something important.
AI is good enough to generate a first pass for an increasingly large number of projects. An engineer using AI can create an even larger amount.
But muscle memory and expertise are not fixed. They fade over time without use. If you're no longer writing code yourself, you'll get rusty on syntax in the short term. In the long term, you'll get rusty on code structure and layout.
But many engineers are no longer reviewing code either. Reviewing code written by an AI is now the bottleneck, so you're expected to allow AI to review it as well.
So we have a group of people who are no longer engaging in either the writing of the code, or the analysis of the code that's written.
Obviously, this would lead to skill loss.
Many would argue that they're not truly losing skills, because they're more engaged in the grander architecture of the code. To that I would say--your job title says engineer, not architect.
It is probably skill loss on one side and gain on another. In a few years it is going to matter even less if you can code manually. The hard part now is understanding what agents do. How can you gain a bit of confidence in their work? I think it relates to testing, you can trust the parts you test.
i don't really buy the "loosing skills" line. I'm not old enough to remember but it seems like this would be similar to an expert slide rule operator lamenting the skill loss because of using a calculator. A new tool has come out that antiquates some skills but requires a new set of other skills to master.
Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress.
Jarred is not a mathematician, and not an expert on the Riemann conjecture. What new skill is needed to tell the LLM to believe in itself? What part of that work couldn't have been done by a ten year old?
Comments
" skill decline is real"
Is this -> More people that are not devs, are using AI to create things, and are skilling up to a mediocre level?
or
Is this -> Existing experts, are actually loosing there skills? Using AI is reducing someone already known skills.
I think as an existing expert in my field (robot devops) what I feel most strongly is not that I'm losing my skills but rather than I'm losing my nerve.
Like, problems that I would previously dig into on my own, poking at this and that log file to try to understand what happened to get a system into a particular state, now claude is mediating most of those kinds of interactions and it is the one doing the first pass surfacing of "okay I discovered X, Y, and Z things that are hinky, and I'm not totally sure yet what this all means, but let's look together."
Nowadays I feel almost naked looking at a terminal where I'm typing each character myself. A lot of the old instincts around tab completion and grepping through the --help output of every tool, all that stuff has atrophied somewhat. Maybe that is genuine skill decline?
I think this is a really important idea. Good problem solvers, in general, are people who are prepared to go in and poke around, to see what happens when you disable this or add that.
The only way I've found to make juniors develop that instinct is to make them do it until it comes naturally--to continually demand they verify their assumptions, test out theories while debugging. If you don't practice being comfortable with the unfamiliar, you lose the knack.
It's fine, of course, to lose the knack to AI, as long as you'll never need to solve a problem the AI can't (or can't access). I don't think that's a good assumption for everyone to make.
you have to do all this when dealing with AI too. The syntax is a conversation vs a strict grammar but you still have to do all the exploration, verification, and evaluation you always had to do as a developer. You have to know how to explain what you want and know when you have it, that's not always an easy skill to learn though.
Nah, you really just need to tell it "the program broke: <stack trace>". It'll go and poke around for you just fine.
Try it sometime.
The truly wild situation for me is when I have something self contained enough that I can let the bot manage the entire loop unsupervised. Like, here's the patches from the old version, here's the new version, here's the test suite that's got to pass. I'm going to bed now, see you in the morning.
And in the morning, I have patches for the new version that pass the test suite.
What skills are being developed by non-devs prompting AI to churn out code they don't understand? Note that I'm not debating if the code is good or works or whatever, I'm asking if any real skills are being developed merely by prompting AI towards some goal.
my sister and two of her friends put together a small company. She asked me to work on a mobile app for them, like a small niche LMS. I was making progress but fell behind, i have a demanding job/family and she doesnt. She asked if she could try to take it over, i told her about claudecode and then gave her some magic words to get started with. Basically, some framework names and some other technologies involved and some prompt guidance. She completely took off with it and came up with a flow where she chats with ChatGPT and it tells her what to tell ClaudeCode, she copy/pastes into claudecode does it's thing.
How could it possibly work? Except she got through setting up her own Apple dev account, all the certificates, subscription entitlement, revenuecat integration, TestFlight, all the paperwork that goes along with a listing and all of that in about 3 weeks. She submitted her app to the AppStore for review last friday.
She didn't learn any coding but she sure as hell produced a pretty good mobile app with a subscription payment system all on her own with the help of ChatGPT and ClaudeCode. She's also producing different tools that run on her website which is hosted on squarespace. I have no idea how she's doing it but she has all these little questionairre and decision tree pages backed by google sheets. She's the second non-coder i've seen really blossom with AI and just run with it building stuff she's always wanted but never knew how.
I hope she has insurance when that db gets hacked or some other code disaster happens.
I don't think it is zero.
If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero.
Its just what they are learning, is not as much as they think, and really random, not-structured. Like reading just a few random chapter of a book across multiple classes. They get a smattering of ad-hoc tidbits of knowledge.
I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.
Maybe not actually absolutely zero, but it's much closer to zero than what they would learn creating the app themselves
I think the knowledge being absorbed is 'how to use AI to make apps', not necessarily anything about the code itself. I've seen lots of people who are very effective with AI suddenly poleaxed in interviews without it. Going from 'able to explain exactly what needs to happen' to 'staring at a blank page with no ideas'.
It absolutely is. It removes a lot of friction from pathways that kept previous knowledge firm.
This isn't unique to AI. When calculators came around, people had a new tool that decreased the need to do arithmetic in your head. This has lead to a decline in the percentage of people who knew arithmetic, but can no longer do it adequately without the external tool. Another example would be assembly skills declining amongst software engineers after compiled languages become dominant.
What I would argue is unique to AI is two-fold. The first being that the scale of things being automated is enormous. Keeping with just software, AI isn't only a tool that decreases the need to write code, it's also a tool that decreases the need to do your own research, debugging, system design, version control, etc. The list of tasks a regular engineer is responsible for that have not been consumed by AI is small. To make matters more frightening, this is just a single industry. The same applies to multiple other knowledge heavy fields (e.g. mathematics).
The second thing I'd argue is unique about AI is the addictive nature of it. These LLMs are deliberately coerced into being sycophants that blow smoke up your ass unless you deliberately tell the model not too (and it may still do it anyways!). A calculator didn't compliment your insight for asking it to calculate the square root of -1.
Furthermore, many people have talked about the resemblance of LLMs to slot machines--there's a feeling that anything could be possible if you only prompt the model correctly. Your first prompt is wildly off the mark, but your next prompt is better though still not quite right, so you continue forward. You keep iterating your instructions, your word choices, your tone, all in search of the desired response. In this way, an LLM is a slot machine, your prompt the lever you pull, and the output is the jackpot.
All this taken together, we have a machine that decreases the need to engage with the critical faculties of your own brain, which has been trained to keep your attention through flattery, and whose probabilistic nature preys on our love for gambling.
Why not both?
I've got a friend whose firm has started using 'cognitive surrender' to describe the habit of delegating thinking to the LLM in the way that we used to skim code reviews and say 'LGTM' while missing something important.
Both?
AI is good enough to generate a first pass for an increasingly large number of projects. An engineer using AI can create an even larger amount.
But muscle memory and expertise are not fixed. They fade over time without use. If you're no longer writing code yourself, you'll get rusty on syntax in the short term. In the long term, you'll get rusty on code structure and layout.
But many engineers are no longer reviewing code either. Reviewing code written by an AI is now the bottleneck, so you're expected to allow AI to review it as well.
So we have a group of people who are no longer engaging in either the writing of the code, or the analysis of the code that's written.
Obviously, this would lead to skill loss.
Many would argue that they're not truly losing skills, because they're more engaged in the grander architecture of the code. To that I would say--your job title says engineer, not architect.
It is probably skill loss on one side and gain on another. In a few years it is going to matter even less if you can code manually. The hard part now is understanding what agents do. How can you gain a bit of confidence in their work? I think it relates to testing, you can trust the parts you test.
How can you understand them when everything changes again 6 months from now?
i don't really buy the "loosing skills" line. I'm not old enough to remember but it seems like this would be similar to an expert slide rule operator lamenting the skill loss because of using a calculator. A new tool has come out that antiquates some skills but requires a new set of other skills to master.
But it genuinely doesn't require much skill. That's the main selling point, in fact.
Look at the prompts used to make progress on the Riemann conjecture: https://www.anthropic.com/research/riemann-zeta
Jarred is not a mathematician, and not an expert on the Riemann conjecture. What new skill is needed to tell the LLM to believe in itself? What part of that work couldn't have been done by a ten year old?
Definitely the latter.