The reason to have a dedicated "software design phase" is to encourage invention of "nobody can do it but us" - secret sauces which can make you (or your company) super rich (i.e., algorithms in Oracle Database, Google's algorithms, etc.). These things cannot be invented as "ongoing and continual process".
I know that my examples are based on system software development, but I have no reason to believe they cannot be applied also for web development.
That's more R&D than part of the day-to-day engineering process, though.
Anything that's going to be secret sauce is going to need a lot of prototyping and experimentation, and might not even yield usable results. Oftentimes, it isn't even possible to make an estimate as to the complexity of what it is you are building -- all you can do is timebox it.
Mixing that in with normal day-to-day development feels, to my brain, like mixing concerns.
Once those prototypes and experiments start to produce tangible benefits, then incorporating them just becomes another story in the sprint.
There are two kinds of innovations: "initial" and "incremental" innovation. What you just described is "initial" innovation (some people also call it "radical innovation"). And the process you described, I think it is called "skunk work" (prototypes and experiments) which is only one way to achieve "initial" innovation.
I think that "incremental" innovations are very important for companies and sometimes a "incremental" innovation can end up being a "radical" innovation.
The reason to have a dedicated "software design phase" is to encourage invention of "nobody can do it but us" - secret sauces which can make you (or your company) super rich (i.e., algorithms in Oracle Database, Google's algorithms, etc.). These things cannot be invented as "ongoing and continual process".
That doesn't match my experience - and I'm not sure I understand the logic. Why would these algorithms be impossible to development during an incremental process?
From my perspective having a continual software design process, rather than a phased approach, gives me more opportunities for innovative design - not less. We end up doing more software design work that way, since we have the opportunity to add "secret sauce" all of the time - rather than just in some one-off section.
I find I get more innovation out of teams that develop incrementally. That is, to quite a large extent, why we develop that way.
Comments
The reason to have a dedicated "software design phase" is to encourage invention of "nobody can do it but us" - secret sauces which can make you (or your company) super rich (i.e., algorithms in Oracle Database, Google's algorithms, etc.). These things cannot be invented as "ongoing and continual process".
I know that my examples are based on system software development, but I have no reason to believe they cannot be applied also for web development.
That's more R&D than part of the day-to-day engineering process, though.
Anything that's going to be secret sauce is going to need a lot of prototyping and experimentation, and might not even yield usable results. Oftentimes, it isn't even possible to make an estimate as to the complexity of what it is you are building -- all you can do is timebox it.
Mixing that in with normal day-to-day development feels, to my brain, like mixing concerns.
Once those prototypes and experiments start to produce tangible benefits, then incorporating them just becomes another story in the sprint.
There are two kinds of innovations: "initial" and "incremental" innovation. What you just described is "initial" innovation (some people also call it "radical innovation"). And the process you described, I think it is called "skunk work" (prototypes and experiments) which is only one way to achieve "initial" innovation.
I think that "incremental" innovations are very important for companies and sometimes a "incremental" innovation can end up being a "radical" innovation.
The reason to have a dedicated "software design phase" is to encourage invention of "nobody can do it but us" - secret sauces which can make you (or your company) super rich (i.e., algorithms in Oracle Database, Google's algorithms, etc.). These things cannot be invented as "ongoing and continual process".
That doesn't match my experience - and I'm not sure I understand the logic. Why would these algorithms be impossible to development during an incremental process?
From my perspective having a continual software design process, rather than a phased approach, gives me more opportunities for innovative design - not less. We end up doing more software design work that way, since we have the opportunity to add "secret sauce" all of the time - rather than just in some one-off section.
I find I get more innovation out of teams that develop incrementally. That is, to quite a large extent, why we develop that way.