I've read C10K years ago. I don't consider it a useful educational document. It doesn't go deep enough into the subject and it doesn't describe well enough why things like threading don't work well. I all had to find that out through experience (of writing a web server myself), and even now some things are still blurry. C10K is very old, it doesn't describe recent advances in threading implementations. Why are threads unscalable? Is it because of context switching overhead? Because of the VM addresses required for the thread stacks? Because of frequent CPU cache flushes? C10K doesn't go into those kind of details and it doesn't look like many people know the answer. I suppose I can find out by studying the kernel but frankly I have better things to do with my time, like actually writing software and making a living.
Furthermore C10K is not the complete picture. It describes only connection management, not what you actually do with the connection. The latter plays a non-trivial role in actual scalability.
Nobody said C10K is the end-all-be-all on scaling, nor should it be. It is also false that it doesn't present any arguments for why threads don't scale well. According to the change log the latest change was 2011 which added information on nginx, but the last change prior to that was 2006. That version of the document talks about memory limitations on using threads because of the stack size. While not extensive it also includes a quote from Drepper about context switches. It even points to documents that are pro-threading. My point is that it's been around for 12 years and is easily found and read by anyone. It also contains a plethora if links to more information on the subject. To claim that C10K is not a useful educational document is absurd. But C10K is one document, there are numerous ones out there, a google search away, our knowledge of scaling has anything but stagnated in the last 12 years. Your options aren't C10K or read the kernel source for your favorite OS.
If you're going to argue that those with knowledge need to distribute that knowledge better, that's fine. Knowledge can almost always be distributed better, perhaps someone could make a nice centralized website that has better information than highscalability.com. But at the same time you've just told me that a document that is a great introductory resource on scaling connection handling is not a "useful educational document". You may have better things to do with your time than read kernel source, but is your time so precious you can't do some google searches? Perhaps read an industrial white paper or academic paper on the subject of scalability? You can write all the software you want but if you're ignorant of how to overcome scalability problems are you accomplishing much? And if you're doing tests and learning about what scales but keeping it to yourself you are just as culpable of not educating people.
Comments
I've read C10K years ago. I don't consider it a useful educational document. It doesn't go deep enough into the subject and it doesn't describe well enough why things like threading don't work well. I all had to find that out through experience (of writing a web server myself), and even now some things are still blurry. C10K is very old, it doesn't describe recent advances in threading implementations. Why are threads unscalable? Is it because of context switching overhead? Because of the VM addresses required for the thread stacks? Because of frequent CPU cache flushes? C10K doesn't go into those kind of details and it doesn't look like many people know the answer. I suppose I can find out by studying the kernel but frankly I have better things to do with my time, like actually writing software and making a living.
Furthermore C10K is not the complete picture. It describes only connection management, not what you actually do with the connection. The latter plays a non-trivial role in actual scalability.
Nobody said C10K is the end-all-be-all on scaling, nor should it be. It is also false that it doesn't present any arguments for why threads don't scale well. According to the change log the latest change was 2011 which added information on nginx, but the last change prior to that was 2006. That version of the document talks about memory limitations on using threads because of the stack size. While not extensive it also includes a quote from Drepper about context switches. It even points to documents that are pro-threading. My point is that it's been around for 12 years and is easily found and read by anyone. It also contains a plethora if links to more information on the subject. To claim that C10K is not a useful educational document is absurd. But C10K is one document, there are numerous ones out there, a google search away, our knowledge of scaling has anything but stagnated in the last 12 years. Your options aren't C10K or read the kernel source for your favorite OS.
If you're going to argue that those with knowledge need to distribute that knowledge better, that's fine. Knowledge can almost always be distributed better, perhaps someone could make a nice centralized website that has better information than highscalability.com. But at the same time you've just told me that a document that is a great introductory resource on scaling connection handling is not a "useful educational document". You may have better things to do with your time than read kernel source, but is your time so precious you can't do some google searches? Perhaps read an industrial white paper or academic paper on the subject of scalability? You can write all the software you want but if you're ignorant of how to overcome scalability problems are you accomplishing much? And if you're doing tests and learning about what scales but keeping it to yourself you are just as culpable of not educating people.