Why don't you just lock each connection handler to one thread at a time and dispatch events on a thread pool ? That way connection level events are always synchronous, but event handlers are spread over the thread pool, you get optimal load balancing because events fill the pool (no processes) and thread pool can use it's own logic to grow if one channel handler used blocking IO and is blocking the a pool thread.
This is pretty much what netty does with OrderedMemoryAwareThreadPoolExecutor ?
It does have advantages from code-organisation, but not from all-out performance.
One of the complexities of this is ensuring that a selector doesn't return a handle as being read/writable when another thread is still executing a previous trigger.
Comments
Why don't you just lock each connection handler to one thread at a time and dispatch events on a thread pool ? That way connection level events are always synchronous, but event handlers are spread over the thread pool, you get optimal load balancing because events fill the pool (no processes) and thread pool can use it's own logic to grow if one channel handler used blocking IO and is blocking the a pool thread.
This is pretty much what netty does with OrderedMemoryAwareThreadPoolExecutor ?
http://netty.io/docs/stable/api/org/jboss/netty/handler/exec...
It does have advantages from code-organisation, but not from all-out performance.
One of the complexities of this is ensuring that a selector doesn't return a handle as being read/writable when another thread is still executing a previous trigger.