They go nowhere because people are using wildly different definitions and contexts. There's one already in here about how ai is better than humans at coding.
- Yes llms are better at the mechanics of coding
- no they're not good enough for overall software dev.
- yes, this stuff should've been automated years ago in frameworks and in libs, or through sane programming langs that dealt with memory and logic flow better.
- yes, they're much more useful for documentation, search, etc. than they are at actual coding
- no, your stats aren't useful - 90% of coding is meaningless if you've also massively increased the amount of slop produced
- yes, llms do make you more productive overall, whether it's 10-30% or 1000% is context dependent
- yes and no that AI will change everything; no it doens't make sense to keep comparing pre-ai and post-ai worlds, it is very likely that the gains cancel each other out and we all just move up a layer of abstraction, and end up in a simlar situation to now.
- no we don't know what will actually happen to the job market, things can remain irrational longer than you can remain solvent. We've had the technology to be where are 20-30 years ago if we were focused as a society. Things take time, and real world is complicated.
I'm biased, being a software technical writer and not a dev, but at least from my side of the fence I think LLMs are no better at generating docs than they are at generating code, and might actually be way worse :)
My devs keep throwing Claude-generated writeups at me that look okay at first glance but fall apart when you actually try to follow the instructions they lay out (which makes them useless for anyone who's not already familiar with the product or feature they're describing), and/or have major structural and logical gaps, and/or gloss over messy details in a way that makes the product or feature sound better but doesn't reflect the reality of what we actually built.
Which turns into an ironic ouroboros where I then have to punch my own queries into an LLM and ask it to read our codebase, compare those details against the original writeup, try to decipher what the hell it was trying to say, etc. And even that turns into an extended back-and-forth where my LLM is like "The original writeup is inaccurate; it should say X" and I'm like "Are you sure? That contradicts what I know about related concept Y", and then the LLM is like "Good point, I dug into it more and the real answer is Z." (And even then, god only knows if Z is correct. I still have to get real developer eyeballs on anything before it goes live.)
Documentation was difficult to keep in sync with code because the tools didn't operate at a natural language level. Things like refactoring names, or terminology, or concept changes (foo is now fooGroup that contains bar) were tedious.
The granularity at which documentation parity is kept makes a big difference.
For example, brekaign things down into explicit assertion blocks in the code, and a link to the documentaiton that covers that assertion. So block-to-block tracking works well. Tracking completely separate documentation to independent codebase... idk I haven't tried it but my guess it that the context windows are too small for that.
But minor things like making sure there are unique error codes, that the codes somewhat make sense, etc. are all helpful. If nothing else, the model being stupid and acting as a rubber duck for pair-programming is useful for knowing what to document given a different perspective.
"this stuff should've been automated years ago in frameworks and in libs, or through sane programming langs that dealt with memory and logic flow better."
70 years of working on the 'how do I reuse code?'. And the solution just takes all the electricity in the world.
Comments
This conversation again?
They go nowhere because people are using wildly different definitions and contexts. There's one already in here about how ai is better than humans at coding.
- Yes llms are better at the mechanics of coding
- no they're not good enough for overall software dev.
- yes, this stuff should've been automated years ago in frameworks and in libs, or through sane programming langs that dealt with memory and logic flow better.
- yes, they're much more useful for documentation, search, etc. than they are at actual coding
- no, your stats aren't useful - 90% of coding is meaningless if you've also massively increased the amount of slop produced
- yes, llms do make you more productive overall, whether it's 10-30% or 1000% is context dependent
- yes and no that AI will change everything; no it doens't make sense to keep comparing pre-ai and post-ai worlds, it is very likely that the gains cancel each other out and we all just move up a layer of abstraction, and end up in a simlar situation to now.
- no we don't know what will actually happen to the job market, things can remain irrational longer than you can remain solvent. We've had the technology to be where are 20-30 years ago if we were focused as a society. Things take time, and real world is complicated.
I'm biased, being a software technical writer and not a dev, but at least from my side of the fence I think LLMs are no better at generating docs than they are at generating code, and might actually be way worse :)
My devs keep throwing Claude-generated writeups at me that look okay at first glance but fall apart when you actually try to follow the instructions they lay out (which makes them useless for anyone who's not already familiar with the product or feature they're describing), and/or have major structural and logical gaps, and/or gloss over messy details in a way that makes the product or feature sound better but doesn't reflect the reality of what we actually built.
Which turns into an ironic ouroboros where I then have to punch my own queries into an LLM and ask it to read our codebase, compare those details against the original writeup, try to decipher what the hell it was trying to say, etc. And even that turns into an extended back-and-forth where my LLM is like "The original writeup is inaccurate; it should say X" and I'm like "Are you sure? That contradicts what I know about related concept Y", and then the LLM is like "Good point, I dug into it more and the real answer is Z." (And even then, god only knows if Z is correct. I still have to get real developer eyeballs on anything before it goes live.)
Documentation was difficult to keep in sync with code because the tools didn't operate at a natural language level. Things like refactoring names, or terminology, or concept changes (foo is now fooGroup that contains bar) were tedious.
The granularity at which documentation parity is kept makes a big difference.
For example, brekaign things down into explicit assertion blocks in the code, and a link to the documentaiton that covers that assertion. So block-to-block tracking works well. Tracking completely separate documentation to independent codebase... idk I haven't tried it but my guess it that the context windows are too small for that.
But minor things like making sure there are unique error codes, that the codes somewhat make sense, etc. are all helpful. If nothing else, the model being stupid and acting as a rubber duck for pair-programming is useful for knowing what to document given a different perspective.
"this stuff should've been automated years ago in frameworks and in libs, or through sane programming langs that dealt with memory and logic flow better."
70 years of working on the 'how do I reuse code?'. And the solution just takes all the electricity in the world.