So the server has multiple threads. If a handler blocks in one thread, another thread can pick up incoming requests. So far, so good. In return for needing to carefully synchronise access to shared state, we get to efficiently share that state (even if its just a hot cache of secure session cookies - things you don’t want to be validating every incoming request etc) between many threads and multiplex incoming requests between them.
Sharing state is bad, m-kay? Allow Node to do it's thing (So the server has multiple threads. If a handler blocks in one thread, another thread can pick up incoming requests.).
It's aggrieving that this model requires any given handler to be able to service any given request, tbh. Shared state is a folly. A serializing token scheme might work well: if a request fails to find the data local to it's core, it passes a serializing token of the request around the ring of handlers, asking either a, for the required data, or b, take the token and run the data.
Serializing tokens are a concept Matt Dillon spoke of often at the inception of DragonflyBSD; much like locks, except that ownership is not relinquished, someone always hold the token, but instead phase changed, yielded to another. it's a responsive less a stateful ownership.
Sadly that token ownership negotiation requires some kind of interruption in the currently-occupied worker thread: if that thread could be interrupted to do other things, this serializing token negotiation might be an acceptable argument (ending with a) no, i'm busy using that set, b) sorry, i had the data and was free, so i completed it, or c) here's the data, i'm busy and not using it). But it does still require thread-interruption. If the worker thread can yield frequently and resume, finding the answer might be a small enough invisible enough calculation to help plaster over there being interruptions altogether; that's essentially the hope. The result would be the mating of green threading to with location aware latency aware multi-processing.
A key thing I've learned on my travels is how critical shared state is to performance.
At its crudest, lots of web frameworks read a request, wait to receive it all, dispatch it to a handler, gather all the output from the handler, buffered, and then write it.
This is great from the code organization perspective.
Hellepoll is so very much faster because it streams the requests.
Without a green threading approach, this necessarily makes the code slightly uglier, and much more for the coder to juggle.
Of course there are at least three layers in an event loop framework:
* the programmer making the event loop itself who has to juggle everything;
* the prorgammer making the various protocol handlers who has to understand everything too but fits the framework conventions rather than creates them;
* finally the programmer who is using the framework who gets to use the utility and decoration and wrappings made by the protocol programmer and who hopefully doesn't have to understand in anything but the broadest terms the way the framework ticks.
I'm a big fan of Clojure and STM, but that's correctness over performance. I would hope whoever makes some other platform puts as much effort into hiding and protecting the inner shared state that is so critical to performance.
Comments
So the server has multiple threads. If a handler blocks in one thread, another thread can pick up incoming requests. So far, so good. In return for needing to carefully synchronise access to shared state, we get to efficiently share that state (even if its just a hot cache of secure session cookies - things you don’t want to be validating every incoming request etc) between many threads and multiplex incoming requests between them.
Sharing state is bad, m-kay? Allow Node to do it's thing (So the server has multiple threads. If a handler blocks in one thread, another thread can pick up incoming requests.).
It's aggrieving that this model requires any given handler to be able to service any given request, tbh. Shared state is a folly. A serializing token scheme might work well: if a request fails to find the data local to it's core, it passes a serializing token of the request around the ring of handlers, asking either a, for the required data, or b, take the token and run the data.
Serializing tokens are a concept Matt Dillon spoke of often at the inception of DragonflyBSD; much like locks, except that ownership is not relinquished, someone always hold the token, but instead phase changed, yielded to another. it's a responsive less a stateful ownership.
Sadly that token ownership negotiation requires some kind of interruption in the currently-occupied worker thread: if that thread could be interrupted to do other things, this serializing token negotiation might be an acceptable argument (ending with a) no, i'm busy using that set, b) sorry, i had the data and was free, so i completed it, or c) here's the data, i'm busy and not using it). But it does still require thread-interruption. If the worker thread can yield frequently and resume, finding the answer might be a small enough invisible enough calculation to help plaster over there being interruptions altogether; that's essentially the hope. The result would be the mating of green threading to with location aware latency aware multi-processing.
Tokens are very interesting.
A key thing I've learned on my travels is how critical shared state is to performance.
At its crudest, lots of web frameworks read a request, wait to receive it all, dispatch it to a handler, gather all the output from the handler, buffered, and then write it.
This is great from the code organization perspective.
Hellepoll is so very much faster because it streams the requests.
Without a green threading approach, this necessarily makes the code slightly uglier, and much more for the coder to juggle.
Of course there are at least three layers in an event loop framework:
* the programmer making the event loop itself who has to juggle everything;
* the prorgammer making the various protocol handlers who has to understand everything too but fits the framework conventions rather than creates them;
* finally the programmer who is using the framework who gets to use the utility and decoration and wrappings made by the protocol programmer and who hopefully doesn't have to understand in anything but the broadest terms the way the framework ticks.
I'm a big fan of Clojure and STM, but that's correctness over performance. I would hope whoever makes some other platform puts as much effort into hiding and protecting the inner shared state that is so critical to performance.
Do you have a link describing the "serializing tokens" concept?
I'm looking for other uses of this boost::asio concept of a 'strand'. http://www.boost.org/doc/libs/1_49_0_beta1/doc/html/boost_as... Specifically: a serialization constraint wrapping an arbitrary set of handlers.