There's an old saying attributed to Michael A. Jackson (no, not that Michael Jackson):
The First and Second Rules of Program Optimisation
1. Don’t do it.
2. (For experts only!): Don’t do it yet.
Scaling is similar. If you worry about scaling before you're at scale, you're almost certain to spend most of your effort fixing things that won't actually turn out to be bottlenecks. So, as a few other posters have said, wait until you have scale problems and then address them.
A safer variant of this make a copy of your system and hit with with a simulated load. However, it's hard to do this in a usefully realistic way. Usually, you'll be launching at small scale and growing gradually. In such situations, it's best to launch, observe actual traffic patterns, and model your simulated load on that. To recap:
1. Build your application, following rule 1 (don't worry about scale).
2. Launch.
3. Observe your actual traffic, and use it to build a realistic load generator. (Depending on your application, you may be able to simply grab a day's worth of logs and replay them at high speed.)
4. Run a copy of your system and hit it with 10x your actual load. See what breaks; fix; repeat.
That said, here's one tip that will help a lot. A stateless system is easy to scale -- just run more copies of it. Of course, most interesting systems aren't stateless. But often you can push the state into a database. Then you're writing a stateless server that sits on top of a database. Now your code is easy to scale. You're left with the problem of scaling the database, so: at the outset, choose a database that will scale to your needs. This is nontrivial, but is much easier than scaling your own custom stateful code.
> This is nontrivial, but is much easier than scaling your own custom stateful code.
Yes, this is an importance piece - and disagrees somewhat with your "don't think about scaling yet" point. Probably the most important part of designing for scale is separating the design into stateless, soft-state and durable state pieces. The stateless pieces are trivially easy to scale (with enough money), either by improving efficiency, using bigger machines or using more machines. Soft-state (read caches, write-through caches, etc) are a little bit harder - but still won't need a huge amount of care.
The difficult piece is durable state storage (typically a database). When you are just starting out, a big database of everything is probably good enough. Chose a solid, widely used data store (MySQL, Postgres, Mongo, Oracle, MSSQL, etc) and use it. Do as little in this layer as you can - it's going to be the most expensive and difficult to scale. Put your business logic elsewhere. Protect it from read spikes with a cache. Design your schemas carefully.
Depending on your read/write mix, data volume and the requirements of your application, you can get pretty big (1000s of reads per second, 10s of writes) without any special database hardware or knowledge.
Things to remember:
1) Avoid tight coupling between stateless and durable state parts of your application. There is nothing wrong with running your DB on the same box as your web server when you are small, but don't write code that assumed that architecture.
2) Chose your data model well. Think carefully about the Nouns in your business, and the relationships between those Nouns (much like you would in OO design). Chose the interfaces and relationships between nouns carefully to reduce linkage. Try to keep interfaces clean.
3) Measure, don't assume. Your page loads slowly? Don't throw out Apache and replace it with Nginx. Measure what is taking the time, and concentrate on the slow piece. If your actual web server is slowing you down, then change servers. Most often, though, slowness is going to be either in your database or your application code.
4) Worry about interfaces, objects and design. Don't worry about technology. The latest buzzwords will not save you from bad design practices - and bigger hardware will only save you for so long.
The difficult piece is durable state storage (typically a database). When you are just starting out, a big database of everything is probably good enough. Chose a solid, widely used data store (MySQL, Postgres, Mongo, Oracle, MSSQL, etc) and use it. Do as little in this layer as you can - it's going to be the most expensive and difficult to scale. Put your business logic elsewhere. Protect it from read spikes with a cache. Design your schemas carefully.
I disagree with some of this. In general the worst db scaling I have seen are ones with large numbers of simple queries based on the idea of doing as little in the db as you can. Instead I would suggest two principles regarding making the db a little more scaling-friendly:
1) Everything that needs to be queried together should be queried together. Don't do lots of round trips and simple queries.
2) Don't do stuff in your database that it isn't designed to do. Write good queries, but don't do things like send emails from the db backend.
A corollary here is that you should write your queries with performance in mind but not do too much premature optimization. For example, it's a lot easier to go from a group by to a sparse index scan (using a stored proc or a CTE) than it is to lock yourself into a sparse index scan from the get go.
In general, the four points you mention though are extremely well thought out. Following up on #4, although technology is important sometimes (esp. newly maturing technologies here like Postgres-xc), these are going to be far more useful where an app is well designed than where it is not. After all, if you don't know where the bottlenecks are in your app, you can't put effort into the right places to fix them.
"4) Worry about interfaces, objects and design. Don't worry about technology. The latest buzzwords will not save you from bad design practices - and bigger hardware will only save you for so long."
Excellent advice, this is really what is critical in building a scalable application from a coding perspective
Session locking (tying a user down to a particular instance in a cluster) can be used to mitigate the statless issue, basically if its data that you are ok with losing if one instance of a cluster goes down then it should be fine to hold that state (such as a user's shopping cart, for example), this could be worth the trade-off for the extra overhead of having to persist the data to the DB. Using state has its places for some things and shouldn't be used in other situations so it all depends but I'm not sure the idea of never having state is necessarily a good hard and fast rule.
Comments
There's an old saying attributed to Michael A. Jackson (no, not that Michael Jackson):
Scaling is similar. If you worry about scaling before you're at scale, you're almost certain to spend most of your effort fixing things that won't actually turn out to be bottlenecks. So, as a few other posters have said, wait until you have scale problems and then address them.A safer variant of this make a copy of your system and hit with with a simulated load. However, it's hard to do this in a usefully realistic way. Usually, you'll be launching at small scale and growing gradually. In such situations, it's best to launch, observe actual traffic patterns, and model your simulated load on that. To recap:
1. Build your application, following rule 1 (don't worry about scale).
2. Launch.
3. Observe your actual traffic, and use it to build a realistic load generator. (Depending on your application, you may be able to simply grab a day's worth of logs and replay them at high speed.)
4. Run a copy of your system and hit it with 10x your actual load. See what breaks; fix; repeat.
That said, here's one tip that will help a lot. A stateless system is easy to scale -- just run more copies of it. Of course, most interesting systems aren't stateless. But often you can push the state into a database. Then you're writing a stateless server that sits on top of a database. Now your code is easy to scale. You're left with the problem of scaling the database, so: at the outset, choose a database that will scale to your needs. This is nontrivial, but is much easier than scaling your own custom stateful code.
> This is nontrivial, but is much easier than scaling your own custom stateful code.
Yes, this is an importance piece - and disagrees somewhat with your "don't think about scaling yet" point. Probably the most important part of designing for scale is separating the design into stateless, soft-state and durable state pieces. The stateless pieces are trivially easy to scale (with enough money), either by improving efficiency, using bigger machines or using more machines. Soft-state (read caches, write-through caches, etc) are a little bit harder - but still won't need a huge amount of care.
The difficult piece is durable state storage (typically a database). When you are just starting out, a big database of everything is probably good enough. Chose a solid, widely used data store (MySQL, Postgres, Mongo, Oracle, MSSQL, etc) and use it. Do as little in this layer as you can - it's going to be the most expensive and difficult to scale. Put your business logic elsewhere. Protect it from read spikes with a cache. Design your schemas carefully.
Depending on your read/write mix, data volume and the requirements of your application, you can get pretty big (1000s of reads per second, 10s of writes) without any special database hardware or knowledge.
Things to remember:
1) Avoid tight coupling between stateless and durable state parts of your application. There is nothing wrong with running your DB on the same box as your web server when you are small, but don't write code that assumed that architecture.
2) Chose your data model well. Think carefully about the Nouns in your business, and the relationships between those Nouns (much like you would in OO design). Chose the interfaces and relationships between nouns carefully to reduce linkage. Try to keep interfaces clean.
3) Measure, don't assume. Your page loads slowly? Don't throw out Apache and replace it with Nginx. Measure what is taking the time, and concentrate on the slow piece. If your actual web server is slowing you down, then change servers. Most often, though, slowness is going to be either in your database or your application code.
4) Worry about interfaces, objects and design. Don't worry about technology. The latest buzzwords will not save you from bad design practices - and bigger hardware will only save you for so long.
The difficult piece is durable state storage (typically a database). When you are just starting out, a big database of everything is probably good enough. Chose a solid, widely used data store (MySQL, Postgres, Mongo, Oracle, MSSQL, etc) and use it. Do as little in this layer as you can - it's going to be the most expensive and difficult to scale. Put your business logic elsewhere. Protect it from read spikes with a cache. Design your schemas carefully.
I disagree with some of this. In general the worst db scaling I have seen are ones with large numbers of simple queries based on the idea of doing as little in the db as you can. Instead I would suggest two principles regarding making the db a little more scaling-friendly:
1) Everything that needs to be queried together should be queried together. Don't do lots of round trips and simple queries.
2) Don't do stuff in your database that it isn't designed to do. Write good queries, but don't do things like send emails from the db backend.
A corollary here is that you should write your queries with performance in mind but not do too much premature optimization. For example, it's a lot easier to go from a group by to a sparse index scan (using a stored proc or a CTE) than it is to lock yourself into a sparse index scan from the get go.
In general, the four points you mention though are extremely well thought out. Following up on #4, although technology is important sometimes (esp. newly maturing technologies here like Postgres-xc), these are going to be far more useful where an app is well designed than where it is not. After all, if you don't know where the bottlenecks are in your app, you can't put effort into the right places to fix them.
"4) Worry about interfaces, objects and design. Don't worry about technology. The latest buzzwords will not save you from bad design practices - and bigger hardware will only save you for so long."
Excellent advice, this is really what is critical in building a scalable application from a coding perspective
Session locking (tying a user down to a particular instance in a cluster) can be used to mitigate the statless issue, basically if its data that you are ok with losing if one instance of a cluster goes down then it should be fine to hold that state (such as a user's shopping cart, for example), this could be worth the trade-off for the extra overhead of having to persist the data to the DB. Using state has its places for some things and shouldn't be used in other situations so it all depends but I'm not sure the idea of never having state is necessarily a good hard and fast rule.