I may sound naive here, but: if all you're doing is serving just 184 flat files, then why do you need all this RoR jazz?
Can't a bunch of Apache/Nginx servers behind a load balancer (Varnish, TrafficServer, etc.) handle that just fine?
Throw in a DNS-based scheme for failover/redundancy?
It's risk mitigation... this is one way you can deal with unpredictably large spikes in traffic.... for the NYT - elections probably represent their extreme.
For the regular ebb and flow of data, you can plan, and use varnish/rails/nginx/ all the magic tools you want, and things will work great - but you also have added complexity, and when something goes wrong, the more complex the system, the longer it generally takes to fix. Especially under unexpected heavy load.
Simplifying the system down to flat files and de-coupling the ROR stuff gives you a clear troubleshooting point - if something goes berzerk, you can cut the link between the two and troubleshoot in relative safety before turning replication back on.
I'm rambling - the main point is to reduce the complexity involved in the end-user transaction to be as efficient and fast as possible so you can deal with an unknown load factor coming in on a really important day. Going down that day would be BAD for business.
The load was for election night, which is more than likely a full order of magnitude, or two above their normal traffic levels. So their normal setup of RoR is fine for normal amounts of load, but would fail under that expected spike. So they used all their page-creation code in rails, but instead of serving it out like normal, they create the static files, and serve those. You lose out on interactivity, but make up for it in performance.
Comments
I may sound naive here, but: if all you're doing is serving just 184 flat files, then why do you need all this RoR jazz? Can't a bunch of Apache/Nginx servers behind a load balancer (Varnish, TrafficServer, etc.) handle that just fine? Throw in a DNS-based scheme for failover/redundancy?
Ruby on Rails is generating the flat files, while apache et al. are serving them.
It's risk mitigation... this is one way you can deal with unpredictably large spikes in traffic.... for the NYT - elections probably represent their extreme.
For the regular ebb and flow of data, you can plan, and use varnish/rails/nginx/ all the magic tools you want, and things will work great - but you also have added complexity, and when something goes wrong, the more complex the system, the longer it generally takes to fix. Especially under unexpected heavy load.
Simplifying the system down to flat files and de-coupling the ROR stuff gives you a clear troubleshooting point - if something goes berzerk, you can cut the link between the two and troubleshoot in relative safety before turning replication back on.
I'm rambling - the main point is to reduce the complexity involved in the end-user transaction to be as efficient and fast as possible so you can deal with an unknown load factor coming in on a really important day. Going down that day would be BAD for business.
The load was for election night, which is more than likely a full order of magnitude, or two above their normal traffic levels. So their normal setup of RoR is fine for normal amounts of load, but would fail under that expected spike. So they used all their page-creation code in rails, but instead of serving it out like normal, they create the static files, and serve those. You lose out on interactivity, but make up for it in performance.
> I may sound naive here, but: if all you're doing is serving just 184 flat files, then why do you need all this RoR jazz?
Generate the static files: those files contained live election results, so they had to be regenerated as new results came in.