Full rewrites like this are a pain, high risk, and always take longer than expected (feature creep/deployment/etc)
This is accepted wisdom in large parts of the software development world, but I have become sceptical about treating it as a universal rule.
In the end, all software maintenance means rewriting part of the system at some scale, whether it’s a one-line bug fix or a Big Rewrite of the whole world. Both the risks and the potential benefits typically increase with the scale of the job, so how much of the system to change in one go is always a cost/benefit question.
All software maintenance where we delete or modify old code risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code. However, it also gives us a chance to make the new code more systematic and to take into account additional knowledge gained and better tools that have become available since the older version was written. This means the new version also has an opportunity to eliminate subtle bugs and awkward edge cases that have accumulated in the old code.
For me, one of the times when a relatively large rewrite is indicated is when there are big changes to the foundations on which the system or some part of it is built: you need to support new hardware or run on a new operating system, there’s a big update to your programming language available and you want to be able to use it, the protocols or APIs or data models used to interact with other systems have changed. Another is when the original software architecture has been outgrown and technical debt is severely affecting productivity. In this case, it appears that both of those factors may apply, so maybe a wide-reaching update does make sense.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
And with that in mind, I should clarify im not opposed to a complete gutting of a legacy codebase and I'm not afraid of doing it in a full rewrite sense of the word either. I just like to have a strategy and keep master green throughout.
risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code.
This shouldn't be happening in a sufficiently tested or sufficiently understood system and I feel like using it as an excuse to not rewrite is a lack of reasoning skills.
Full rewrites are a waste of time because you end up spending way more time than expected making useless decisions around the edges about APIs that really don't matter that much.
Or you end up finding out your dream design has a fatal flaw. Or you end up just porting a bunch of boiler plate.
Rule of thumb,
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
And a caveat to all this, I know a couple of teams that rewrite every system on a two year schedule. They do this as a a way to stay nimble. I believe their feature set and domain requirements are well known. This sort of setup has a few benefits and traits I like. Most notably the fact design skills are constantly being flexed instead of stagnating. Junior devs are made into seniors with this process.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
Interesting choice of phrase. What would you say is the difference between refactoring and rewriting, assuming we’re talking about keeping the externally observable behaviour of the affected part of the system the same in each case?
This shouldn't be happening in a sufficiently tested or sufficiently understood system
Perhaps, but as systems and the teams that develop them evolve over time, it is all but inevitable that some knowledge and understanding are lost. You’ll never have perfect documentation. You’ll never have tests for every possible case. You’ll never have an unambiguous and readily located comment for every subtle detail in the code that someone seven years ago spent a couple of hours thinking about before they put finger to keyboard, when three other people have since refactored that code, generalised its interface, and fixed a performance glitch in a dependency so some aspects of the original design became unnecessary. Preserving knowledge and understanding as well as possible is one of the challenges facing any team with a long-lived system.
Those losses are a form of technical debt, and as it accumulates, two things happen. First, the original code becomes harder to maintain and develop. Second, the original code becomes more risky to rewrite. So unfortunately, there’s a fairly high probability in practice that a part of a system that has become sufficiently difficult that a large-scale rewrite is being considered will also be a part of the system where the current development team have incomplete understanding of the current implementation.
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
But how do you do a piecemeal refactoring exercise if, say, you’re moving to a new programming language? I had to rewrite a large part of a relatively complicated web UI a few years ago, like for like. Although it had been working reasonably well to that point and the design was quite clean and had stood the test of time, it was built around a Java applet. That design made sense when we were first implementing that system, at a time when JS was slow and many features we take for granted in browsers today weren’t yet widely supported. It was less helpful when the great and the good of the browser world decided that Java applets were a liability and they were going to make them harder and harder to use until eventually they killed them off completely. Now, we did have a clean, careful interface between the applet and the various other parts of the page that were written in JS, so we only had to redo that part of the system. However, we had no choice but to reimplement everything the applet did, using alternatives like JS and SVG, starting from scratch. This is the sort of scenario I was talking about before when I mentioned changes in the foundations of a system.
Comments
Full rewrites like this are a pain, high risk, and always take longer than expected (feature creep/deployment/etc)
This is accepted wisdom in large parts of the software development world, but I have become sceptical about treating it as a universal rule.
In the end, all software maintenance means rewriting part of the system at some scale, whether it’s a one-line bug fix or a Big Rewrite of the whole world. Both the risks and the potential benefits typically increase with the scale of the job, so how much of the system to change in one go is always a cost/benefit question.
All software maintenance where we delete or modify old code risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code. However, it also gives us a chance to make the new code more systematic and to take into account additional knowledge gained and better tools that have become available since the older version was written. This means the new version also has an opportunity to eliminate subtle bugs and awkward edge cases that have accumulated in the old code.
For me, one of the times when a relatively large rewrite is indicated is when there are big changes to the foundations on which the system or some part of it is built: you need to support new hardware or run on a new operating system, there’s a big update to your programming language available and you want to be able to use it, the protocols or APIs or data models used to interact with other systems have changed. Another is when the original software architecture has been outgrown and technical debt is severely affecting productivity. In this case, it appears that both of those factors may apply, so maybe a wide-reaching update does make sense.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
And with that in mind, I should clarify im not opposed to a complete gutting of a legacy codebase and I'm not afraid of doing it in a full rewrite sense of the word either. I just like to have a strategy and keep master green throughout.
This shouldn't be happening in a sufficiently tested or sufficiently understood system and I feel like using it as an excuse to not rewrite is a lack of reasoning skills.
Full rewrites are a waste of time because you end up spending way more time than expected making useless decisions around the edges about APIs that really don't matter that much.
Or you end up finding out your dream design has a fatal flaw. Or you end up just porting a bunch of boiler plate.
Rule of thumb,
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
And a caveat to all this, I know a couple of teams that rewrite every system on a two year schedule. They do this as a a way to stay nimble. I believe their feature set and domain requirements are well known. This sort of setup has a few benefits and traits I like. Most notably the fact design skills are constantly being flexed instead of stagnating. Junior devs are made into seniors with this process.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
Interesting choice of phrase. What would you say is the difference between refactoring and rewriting, assuming we’re talking about keeping the externally observable behaviour of the affected part of the system the same in each case?
This shouldn't be happening in a sufficiently tested or sufficiently understood system
Perhaps, but as systems and the teams that develop them evolve over time, it is all but inevitable that some knowledge and understanding are lost. You’ll never have perfect documentation. You’ll never have tests for every possible case. You’ll never have an unambiguous and readily located comment for every subtle detail in the code that someone seven years ago spent a couple of hours thinking about before they put finger to keyboard, when three other people have since refactored that code, generalised its interface, and fixed a performance glitch in a dependency so some aspects of the original design became unnecessary. Preserving knowledge and understanding as well as possible is one of the challenges facing any team with a long-lived system.
Those losses are a form of technical debt, and as it accumulates, two things happen. First, the original code becomes harder to maintain and develop. Second, the original code becomes more risky to rewrite. So unfortunately, there’s a fairly high probability in practice that a part of a system that has become sufficiently difficult that a large-scale rewrite is being considered will also be a part of the system where the current development team have incomplete understanding of the current implementation.
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
But how do you do a piecemeal refactoring exercise if, say, you’re moving to a new programming language? I had to rewrite a large part of a relatively complicated web UI a few years ago, like for like. Although it had been working reasonably well to that point and the design was quite clean and had stood the test of time, it was built around a Java applet. That design made sense when we were first implementing that system, at a time when JS was slow and many features we take for granted in browsers today weren’t yet widely supported. It was less helpful when the great and the good of the browser world decided that Java applets were a liability and they were going to make them harder and harder to use until eventually they killed them off completely. Now, we did have a clean, careful interface between the applet and the various other parts of the page that were written in JS, so we only had to redo that part of the system. However, we had no choice but to reimplement everything the applet did, using alternatives like JS and SVG, starting from scratch. This is the sort of scenario I was talking about before when I mentioned changes in the foundations of a system.