The reasoning behind not cleaning up your code is that if you wait long enough, the problem will often go away - literally! Technology moves fast enough that your feature is usually obsolete within a year or so; if you're not on the critical path that everyone is building upon, chances are your project will just be canceled and all the time spent polishing your code will be wasted. Better to get stuff out there so you have a better chance of being on that critical path, and dealing with the inevitable messes and complaints of "this is shit code!" later.
This is the "cascade of ADHD teenagers" development methodology, which is much maligned by professional programmers, but actually seems to work quite well. Google, FaceBook, and Twitter all use it to varying extents, and the entire valley startup ecosystem is based around it.
> Technology moves fast enough that your feature is
> usually obsolete within a year or so
That depends on your industry. My company has some core infrastructure running on code written in the 90's when characters were scarce (8-character variable names) and actual loops were even scarcer (goto's instead).
Feature-sets like "processes transactions" never go away just because technology moves forward. Some of that almost feels like you've had "too much of the Kool-aid."
My company barely existed in the 90s - it was some research project at Stanford.
And feature sets like "processes transactions" absolutely do go away because technology (and its applications) moves forwards. This was a large part of the success of MySQL: relational databases started getting used in applications where transactional integrity really wasn't that important, and they were better off working around it (or just showing inconsistent data to the user) rather than paying the price for transactions.
Or look at this recent discussion surrounding Google Megastore:
Megastore suffers a large latency penalty for ACID semantics. In many applications, it's better not to take that latency hit and instead settle for eventual consistency instead.
Arguably, this is just that new domains appear or grow from what used to be niche areas into big ones. The old problem domains requiring ACID transactions still remain.
Financial apps are a different story. I interned on Pay.gov in college, and then spent the first 2 years of my career at a financial software startup. The standards are much different when you're handling money (or avionics, or medical devices, or anything that's "mission critical"). And even in the finance world, the standards vary a lot depending on whether you're doing analytics or transaction processing.
But while that software will never go away, it becomes a smaller part of the whole software industry. Much of the interesting growth is in areas of software that have nothing to do with money, other than being able to make it. It's silly to apply the same standards to software that processes $10M/sec (as one program I once wrote did) as to software that lets you throw virtual sheep at each other.
If you look at some of the big investment banks and hedge funds, they won't bother refactoring; they'll just throwaway and re-write.
Their architecture is designed to be really modular and very loosely coupled. So you there is little value lost in throwing away and re-writing (ie. You won't throw stuff away thats working fine with the studd you want to re-write).
Technology moves fast enough that your feature is usually obsolete within a year or so; if you're not on the critical path that everyone is building upon, chances are your project will just be canceled and all the time spent polishing your code will be wasted. Better to get stuff out there so you have a better chance of being on that critical path, and dealing with the inevitable messes and complaints of "this is shit code!" later.
I cannot disagree more strongly. I'm dealing with a decade old application that's been coded using this philosophy, and there are enough code smells here to make a sewage treatment plant seem fragrant.
There are methods many hundreds of lines long, inside classes that are thousands of lines. There are isNegative() functions that return true for values above ten million. There are hidden dependencies all over the place, as people patched in functionality in the wrong place and no one moved it to the right place later. And, of course, there are no unit tests.
The impact of all this is that even trivial changes take weeks, as the developer has to wander a morass of bloated functions and tangled dependencies in order to just find the source of the issue. The "cascade of ADHD teenagers" can work for a couple months, but its not a long term strategy.
To note, none of your examples use that methodology any more. I'm not sure if Google ever used it. Facebook and Twitter may have been like that at the very beginning, but both exhibit quite disciplined coding standards today.
Google totally uses it now - sometimes. I've been specifically told, by a VP (who was one of the early employees), not to refactor code which very much needed it, because the engineering resources were needed elsewhere. There've been other times where I've made the call on my own not to cleanup dirty code, and it was 100% the right decision, because we never touched that code again. There've been still other times that I look at code Jeff Dean or Craig Silverstein wrote and say "This is a piece of shit" - encapsulation violations all over the place, an API that encourages you to do the slow thing (after micro-optimizing for efficiency everywhere else), complicated object ownership semantics. And then I remember that they probably wrote it in a day, for requirements very different than we're facing now, and cut them some slack.
The operative word there is "sometimes". It's a very useful skill to be able to make a judgment call and say "This code is peripheral; we're unlikely to touch it again" or "This code is temporary; we'll replace it in 3 months" or "This code is critical; it better be rock solid, because 500 developers will be changing it every day and learning how to program from it."
The point the article makes is that a manager doesn't have all the information, because they're not the ones who look at the code all day. And the point I'm making is that a developer doesn't have all the information either, because they're often not aware of the opportunity cost of fixing that code. What other features could you have implemented in the time it takes to write those unit tests? What other markets could you tap instead of fixing your integer overflows?
There's a peculiar kind of survivorship bias when looking at code. The fact that you're being paid to look at it means that it was successful enough to actually get users. Usually, that means some sort of trade-offs to actually get to market; the people who spend time writing unit tests and figuring out the perfect design get beaten by competitors who don't (trust me, this happened to my first startup). Thus, all code sucks, and all maintenance programmers grumble, and we'll continue living with shitty software because that shitty software satisfies users better than perfect software.
The fallacy here is that cleaning up your code, doing things right, and writing unit tests slows down your development speed. You could be implementing new features, after all.
But what you will (or would) find if you did take the time to do it right is that it speeds up your development speed tremendously.
I am not talking about proof of concept implementations and skeleton apps, rapid prototyping - these all have their place.
I don't want to write the next Twitter fail whale server app - thanks very much. As for Facebook, they obviously have a rock solid backbone infrastructure. Extremely impressive, and I am willing to bet that it didn't get there by adding hacks upon hacks - to the contrary.
I'm working on code that's over 10 years old (Java) and it's grown organically throughout that period, meandering through whatever technology is trendy (Servlets -> JSP -> JSF -> Dojo).
The problems have not gone away and are, in fact, getting progressively worse. On the other hand, cleaning up the code is a massive effort with little or no perceived benefit to the customer so it won't get done :(
Hm, my understanding from poorly-recalled internet surfing was that Google had a very strong culture of code reviews. But I suppose as a former (current?) Google employee you know better.
Google has a strong culture of code reviews, but that's not mutually exclusive to accepting an expedient but ugly solution and moving on. Most code reviewers are sensitive to pressure to launch; in rare occasions, you'll see something held up while a bunch of architectural decisions are redone, but usually the point of a code review is to fix obviously wrong or easily fixed issues, not to make massive changes that will block a launch.
Comments
The reasoning behind not cleaning up your code is that if you wait long enough, the problem will often go away - literally! Technology moves fast enough that your feature is usually obsolete within a year or so; if you're not on the critical path that everyone is building upon, chances are your project will just be canceled and all the time spent polishing your code will be wasted. Better to get stuff out there so you have a better chance of being on that critical path, and dealing with the inevitable messes and complaints of "this is shit code!" later.
This is the "cascade of ADHD teenagers" development methodology, which is much maligned by professional programmers, but actually seems to work quite well. Google, FaceBook, and Twitter all use it to varying extents, and the entire valley startup ecosystem is based around it.
Feature-sets like "processes transactions" never go away just because technology moves forward. Some of that almost feels like you've had "too much of the Kool-aid."
My company barely existed in the 90s - it was some research project at Stanford.
And feature sets like "processes transactions" absolutely do go away because technology (and its applications) moves forwards. This was a large part of the success of MySQL: relational databases started getting used in applications where transactional integrity really wasn't that important, and they were better off working around it (or just showing inconsistent data to the user) rather than paying the price for transactions.
Or look at this recent discussion surrounding Google Megastore:
http://glinden.blogspot.com/2011/02/comparing-google-megasto...
Megastore suffers a large latency penalty for ACID semantics. In many applications, it's better not to take that latency hit and instead settle for eventual consistency instead.
Arguably, this is just that new domains appear or grow from what used to be niche areas into big ones. The old problem domains requiring ACID transactions still remain.
I meant transactions as in actual exchanges of money, not as in the computer science / database term.
Financial apps are a different story. I interned on Pay.gov in college, and then spent the first 2 years of my career at a financial software startup. The standards are much different when you're handling money (or avionics, or medical devices, or anything that's "mission critical"). And even in the finance world, the standards vary a lot depending on whether you're doing analytics or transaction processing.
But while that software will never go away, it becomes a smaller part of the whole software industry. Much of the interesting growth is in areas of software that have nothing to do with money, other than being able to make it. It's silly to apply the same standards to software that processes $10M/sec (as one program I once wrote did) as to software that lets you throw virtual sheep at each other.
I also think that it's silly to assume that all software follows the same rules as software that allows you to throw virtual sheep at each other.
In the "enterprise" environment, at least the ones I've worked in, this is definitely not the case. But it certainly should be!
If you look at some of the big investment banks and hedge funds, they won't bother refactoring; they'll just throwaway and re-write.
Their architecture is designed to be really modular and very loosely coupled. So you there is little value lost in throwing away and re-writing (ie. You won't throw stuff away thats working fine with the studd you want to re-write).
Technology moves fast enough that your feature is usually obsolete within a year or so; if you're not on the critical path that everyone is building upon, chances are your project will just be canceled and all the time spent polishing your code will be wasted. Better to get stuff out there so you have a better chance of being on that critical path, and dealing with the inevitable messes and complaints of "this is shit code!" later.
I cannot disagree more strongly. I'm dealing with a decade old application that's been coded using this philosophy, and there are enough code smells here to make a sewage treatment plant seem fragrant.
There are methods many hundreds of lines long, inside classes that are thousands of lines. There are isNegative() functions that return true for values above ten million. There are hidden dependencies all over the place, as people patched in functionality in the wrong place and no one moved it to the right place later. And, of course, there are no unit tests.
The impact of all this is that even trivial changes take weeks, as the developer has to wander a morass of bloated functions and tangled dependencies in order to just find the source of the issue. The "cascade of ADHD teenagers" can work for a couple months, but its not a long term strategy.
To note, none of your examples use that methodology any more. I'm not sure if Google ever used it. Facebook and Twitter may have been like that at the very beginning, but both exhibit quite disciplined coding standards today.
Google totally uses it now - sometimes. I've been specifically told, by a VP (who was one of the early employees), not to refactor code which very much needed it, because the engineering resources were needed elsewhere. There've been other times where I've made the call on my own not to cleanup dirty code, and it was 100% the right decision, because we never touched that code again. There've been still other times that I look at code Jeff Dean or Craig Silverstein wrote and say "This is a piece of shit" - encapsulation violations all over the place, an API that encourages you to do the slow thing (after micro-optimizing for efficiency everywhere else), complicated object ownership semantics. And then I remember that they probably wrote it in a day, for requirements very different than we're facing now, and cut them some slack.
The operative word there is "sometimes". It's a very useful skill to be able to make a judgment call and say "This code is peripheral; we're unlikely to touch it again" or "This code is temporary; we'll replace it in 3 months" or "This code is critical; it better be rock solid, because 500 developers will be changing it every day and learning how to program from it."
The point the article makes is that a manager doesn't have all the information, because they're not the ones who look at the code all day. And the point I'm making is that a developer doesn't have all the information either, because they're often not aware of the opportunity cost of fixing that code. What other features could you have implemented in the time it takes to write those unit tests? What other markets could you tap instead of fixing your integer overflows?
There's a peculiar kind of survivorship bias when looking at code. The fact that you're being paid to look at it means that it was successful enough to actually get users. Usually, that means some sort of trade-offs to actually get to market; the people who spend time writing unit tests and figuring out the perfect design get beaten by competitors who don't (trust me, this happened to my first startup). Thus, all code sucks, and all maintenance programmers grumble, and we'll continue living with shitty software because that shitty software satisfies users better than perfect software.
Suit yourself if you want to write crap code.
The fallacy here is that cleaning up your code, doing things right, and writing unit tests slows down your development speed. You could be implementing new features, after all.
But what you will (or would) find if you did take the time to do it right is that it speeds up your development speed tremendously.
I am not talking about proof of concept implementations and skeleton apps, rapid prototyping - these all have their place.
I don't want to write the next Twitter fail whale server app - thanks very much. As for Facebook, they obviously have a rock solid backbone infrastructure. Extremely impressive, and I am willing to bet that it didn't get there by adding hacks upon hacks - to the contrary.
I'm working on code that's over 10 years old (Java) and it's grown organically throughout that period, meandering through whatever technology is trendy (Servlets -> JSP -> JSF -> Dojo).
The problems have not gone away and are, in fact, getting progressively worse. On the other hand, cleaning up the code is a massive effort with little or no perceived benefit to the customer so it won't get done :(
Hm, my understanding from poorly-recalled internet surfing was that Google had a very strong culture of code reviews. But I suppose as a former (current?) Google employee you know better.
Google has a strong culture of code reviews, but that's not mutually exclusive to accepting an expedient but ugly solution and moving on. Most code reviewers are sensitive to pressure to launch; in rare occasions, you'll see something held up while a bunch of architectural decisions are redone, but usually the point of a code review is to fix obviously wrong or easily fixed issues, not to make massive changes that will block a launch.