Web servers such as lighttpd and Apache are blazingly fast at serving static pages. Reading a file from disk and sending its contents back to the browser is as simple as it gets.
Except that it's not. One of the reason the classic web servers are so amazingly fast with static files is that they, along with the operating systems they run on, have spend significant time trying to optimize this process. For linux, for example, look at the sendfile(2) system call, or the TCP_CORK socket option. These were expressly designed to permit a userspace program to get a file through the kernel and onto the wire with as few CPU cycles and memory copies as possible.
One of the most frustrating things about the Web 2.0 crowd (from the perspective of curmudgeons like me) is that they really don't have a clue about any of this complexity. They just figure that they'll stuff everything into a DJango/Rails/whatever request and scale up later. Then when they run into trouble, they end up turning to tools like Apache as black boxes and designing Rube Goldberg apparatii around them when they really should be looking at the problem more directly.
Really, folks: those low level APIs are your friends. They're not nearly as scary as they look. Even if you end up with an off-the-shelf solution, knowledge of this stuff can only be good for you.
Comments
Web servers such as lighttpd and Apache are blazingly fast at serving static pages. Reading a file from disk and sending its contents back to the browser is as simple as it gets.
Except that it's not. One of the reason the classic web servers are so amazingly fast with static files is that they, along with the operating systems they run on, have spend significant time trying to optimize this process. For linux, for example, look at the sendfile(2) system call, or the TCP_CORK socket option. These were expressly designed to permit a userspace program to get a file through the kernel and onto the wire with as few CPU cycles and memory copies as possible.
One of the most frustrating things about the Web 2.0 crowd (from the perspective of curmudgeons like me) is that they really don't have a clue about any of this complexity. They just figure that they'll stuff everything into a DJango/Rails/whatever request and scale up later. Then when they run into trouble, they end up turning to tools like Apache as black boxes and designing Rube Goldberg apparatii around them when they really should be looking at the problem more directly.
Really, folks: those low level APIs are your friends. They're not nearly as scary as they look. Even if you end up with an off-the-shelf solution, knowledge of this stuff can only be good for you.