Very nice. It definitely looks like the scribd guys have a clue.
Scribd is still still running on just one web server. That means they're not at the stage yet where they can just throw machines in to handle growth. That's a pretty significant step. Their one web server just happens to be so damn beefy that it can handle a lot of traffic. It will max out at some point though.
That means they're not at the stage yet where they can just throw machines in to handle growth.
Hmm, why do you say that? They've already got the DB off on other machines, and it sounds very much like they are doing all the things you do to a rails site so that you can scale it by plugging in more app servers. I.e. they already put everything shared into the DB or memcached.
Most people discover fun problems when they split their web servers, no matter how prepared they thought they were. I'm sure they'll handle it well, but it's a significant step.
Yes in theory you're correct. In practice things are usually dirtier.
What if they don't have a load balancer? What if they want redundant load blancers? What if they use round-robin DNS and then need a failover procedure? What if round-robin DNS isn't balanced enough for them? What if logs files need to be merged? What if the web servers aren't 100% identical? Data needs to be sync'd? Load distributed more intelligently? Cron jobs modified? etc, etc.
Perhaps they kept things unusually clean and simple, but most sites require a little finagling when they go to multiple web servers. I'm not saying it's difficult, but I do think it's wise to expect a few bumps.
Comments
Very nice. It definitely looks like the scribd guys have a clue.
Scribd is still still running on just one web server. That means they're not at the stage yet where they can just throw machines in to handle growth. That's a pretty significant step. Their one web server just happens to be so damn beefy that it can handle a lot of traffic. It will max out at some point though.
That means they're not at the stage yet where they can just throw machines in to handle growth.
Hmm, why do you say that? They've already got the DB off on other machines, and it sounds very much like they are doing all the things you do to a rails site so that you can scale it by plugging in more app servers. I.e. they already put everything shared into the DB or memcached.
Most people discover fun problems when they split their web servers, no matter how prepared they thought they were. I'm sure they'll handle it well, but it's a significant step.
It isn't really a significant step. You just plug in another web server and make sure the load balancer and DNS settings are right.
Yes in theory you're correct. In practice things are usually dirtier.
What if they don't have a load balancer? What if they want redundant load blancers? What if they use round-robin DNS and then need a failover procedure? What if round-robin DNS isn't balanced enough for them? What if logs files need to be merged? What if the web servers aren't 100% identical? Data needs to be sync'd? Load distributed more intelligently? Cron jobs modified? etc, etc.
Perhaps they kept things unusually clean and simple, but most sites require a little finagling when they go to multiple web servers. I'm not saying it's difficult, but I do think it's wise to expect a few bumps.