> You develop your first little toy app using Framework X.
> ...
> Then one day you become Twitter, and all hell breaks loose.
Really? Web apps usually become Twitter? I agree that knowing how to scale a web app is important, and that it's not always exactly straightforward, and that it could be made easier with some better tools. Heck, I even gave a talk at PyCon about it (http://www.polimetrix.com/pycon/slides/).
However, statistically speaking, almost zero percent of web apps end up like Twitter. And if they do, they have VCs knocking down their door so hard that they can afford to hire enough engineers and buy enough hardware that scaling won't be an issue for long. That's a problem we'd all love to have.
Not that we shouldn't continue to improve our server-side technology, but 99.9% of my pain developing web apps these days is due to client-side issues. The stuff that runs on the server usually Just Works and is an absolute pleasure to code.
hire enough engineers and buy enough hardware that scaling won't be an issue for long
I don't know about that. Twitter is still having problems with downtime due to scaling. The way I see it: don't completely ignore scaling, but don't focus on it (at all) at the beginning. Keep it in your mind though, and try not to close yourself in, and create something which can't be scaled without significant modifications to services. Do not confuse that with modifications to your code. If you're going to scale large, expect at least one complete re-write, but don't offer services which are incapable of scaling.
Within the context of describing scaling pain, I think it's appropriate, if a bit hyperbolic. Not everybody becomes as big as Twitter, but lots of people get big enough to outgrow their original set of tools.
Agreed. My experience is that optimizing and scaling your app from day one in preparation for Twitter-like traffic is a self-defeating prophecy. Web app scaling is hard, and if you spend so much effort on it from the beginning it will leave little time for actual development, and as a result your web app will most likely fail to become popular.
I'm convinced that scaling is something you need to worry about after you become popular. There's no point to worry about it any earlier.
Comments
> This usually breaks down as follows:
> You develop your first little toy app using Framework X.
> ...
> Then one day you become Twitter, and all hell breaks loose.
Really? Web apps usually become Twitter? I agree that knowing how to scale a web app is important, and that it's not always exactly straightforward, and that it could be made easier with some better tools. Heck, I even gave a talk at PyCon about it (http://www.polimetrix.com/pycon/slides/).
However, statistically speaking, almost zero percent of web apps end up like Twitter. And if they do, they have VCs knocking down their door so hard that they can afford to hire enough engineers and buy enough hardware that scaling won't be an issue for long. That's a problem we'd all love to have.
Not that we shouldn't continue to improve our server-side technology, but 99.9% of my pain developing web apps these days is due to client-side issues. The stuff that runs on the server usually Just Works and is an absolute pleasure to code.
hire enough engineers and buy enough hardware that scaling won't be an issue for long
I don't know about that. Twitter is still having problems with downtime due to scaling. The way I see it: don't completely ignore scaling, but don't focus on it (at all) at the beginning. Keep it in your mind though, and try not to close yourself in, and create something which can't be scaled without significant modifications to services. Do not confuse that with modifications to your code. If you're going to scale large, expect at least one complete re-write, but don't offer services which are incapable of scaling.
Within the context of describing scaling pain, I think it's appropriate, if a bit hyperbolic. Not everybody becomes as big as Twitter, but lots of people get big enough to outgrow their original set of tools.
Agreed. My experience is that optimizing and scaling your app from day one in preparation for Twitter-like traffic is a self-defeating prophecy. Web app scaling is hard, and if you spend so much effort on it from the beginning it will leave little time for actual development, and as a result your web app will most likely fail to become popular.
I'm convinced that scaling is something you need to worry about after you become popular. There's no point to worry about it any earlier.