Well callback style is closest to the metal, greenlets look nicer (at the expense of this opaque dispatch loop) and IMHO both them cause pain over time :/
It's like a deal with the devil for performance... I guess I am disagreeing with the grandparent post. Maybe in Haskell it works (or is free), but C/C++, it works, but you can see you're paying for it.
Well put. You usually pay for it. Not familiar with Haskell too much to tell. It might provide the solution to trick the devil. The devil usually wins the bets though ;-)
> I am not blocking or using event loops, that's the point. When I call read on a socket, the IO manager handles saving my context, doing the async read, yielding, and then restoring context and resuming when there's data to be read.
Doesn't this make it harder to reason about cases where multiple userland threads need to mutate the same state? You can't tell by inspection whether the function you're calling is going to yield somewhere deep inside.
With single threaded callback style you know your thread is the only thing that is executing and won't be interrupted. (Sure, other code can run between now and when your callback is invoked, but in practice I find that to be pretty easy to deal with.)
>Doesn't this make it harder to reason about cases where multiple userland threads need to mutate the same state?
Most languages make it very hard to reason about state in general. Haskell provides STM, so ensuring correct access to shared state is as simple as using an MVar. I absolutely agree that this is an important part of languages making concurrency a priority, they can't just add green threads and pretend that is good enough.
Correct me if I'm wrong but STM doesn't help you if your operations include side effects outside of STM (writing to disk, a database, the network)--exactly what async operations are usually used for. Aren't you back to locking at that point?
I don't understand the problem you're describing. If you're waiting on some external resource then you've yielded to another "thread" in either an event loop or green thread style. Are you suggesting partially modifying some state, then waiting on an external resource before finishing modifying the state? And no other "threads" can touch the state in the mean time? That is going to be a bad thing to do no matter what style you use.
Yes. My point is in the green (or native) thread style, it is not always obvious when you've stumbled into this situation (since you can't tell by looking at a function call whether it's going to yield at some point).
I am not blocking or using event loops, that's the point. When I call read on a socket, the IO manager handles saving my context, doing the async read, yielding, and then restoring context and resuming when there's data to be read. The entire point is that green threads are equivalent to event loops, but taking advantage of abstraction to move the complexity away from the thousands of individual programmers, and onto the language authors.
That link is a good example of exactly why we need to push for languages to take concurrency seriously. That's an example of how a haskell user has to deal with the issue when interfacing with a C lib that rolled their own event loop. For most languages people have to deal with that issue all the time, for their own code, entirely in that one language.
I have no idea why you are talking about pixie dust. My point was that explicitly using event loops instead of abstracting it into a green thread library is dumb, and we need to start having higher expectations from languages rather than accepting backwards shit like node. Not "green threads are magic".
Comments
Well callback style is closest to the metal, greenlets look nicer (at the expense of this opaque dispatch loop) and IMHO both them cause pain over time :/
It's like a deal with the devil for performance... I guess I am disagreeing with the grandparent post. Maybe in Haskell it works (or is free), but C/C++, it works, but you can see you're paying for it.
> It's like a deal with the devil for performance
Well put. You usually pay for it. Not familiar with Haskell too much to tell. It might provide the solution to trick the devil. The devil usually wins the bets though ;-)
Paying for it how? You get a simple model to work with and reason about, and the performance of an async event loop. Where does the problem arise?
OK. Can you block on a sysv semaphore and a socket at the same time? How do you integrate event loops using these two primitives at the same time?
(EDIT: hint your runtime can't, because the kernel doesn't provide a way to do that. I guess I should be more explicit).
> I am not blocking or using event loops, that's the point. When I call read on a socket, the IO manager handles saving my context, doing the async read, yielding, and then restoring context and resuming when there's data to be read.
Doesn't this make it harder to reason about cases where multiple userland threads need to mutate the same state? You can't tell by inspection whether the function you're calling is going to yield somewhere deep inside.
With single threaded callback style you know your thread is the only thing that is executing and won't be interrupted. (Sure, other code can run between now and when your callback is invoked, but in practice I find that to be pretty easy to deal with.)
>Doesn't this make it harder to reason about cases where multiple userland threads need to mutate the same state?
Most languages make it very hard to reason about state in general. Haskell provides STM, so ensuring correct access to shared state is as simple as using an MVar. I absolutely agree that this is an important part of languages making concurrency a priority, they can't just add green threads and pretend that is good enough.
Correct me if I'm wrong but STM doesn't help you if your operations include side effects outside of STM (writing to disk, a database, the network)--exactly what async operations are usually used for. Aren't you back to locking at that point?
I don't understand the problem you're describing. If you're waiting on some external resource then you've yielded to another "thread" in either an event loop or green thread style. Are you suggesting partially modifying some state, then waiting on an external resource before finishing modifying the state? And no other "threads" can touch the state in the mean time? That is going to be a bad thing to do no matter what style you use.
Yes. My point is in the green (or native) thread style, it is not always obvious when you've stumbled into this situation (since you can't tell by looking at a function call whether it's going to yield at some point).
I am not blocking or using event loops, that's the point. When I call read on a socket, the IO manager handles saving my context, doing the async read, yielding, and then restoring context and resuming when there's data to be read. The entire point is that green threads are equivalent to event loops, but taking advantage of abstraction to move the complexity away from the thousands of individual programmers, and onto the language authors.
(I was going to opt out of this, but I can spend another 1/2 hour on it_).
What you are saying is, you're a client of someone else's nonblocking event loop. So all the classical stuff regarding nonblock io loops apply.
E.g: http://talkingcode.co.uk/2008/12/02/haskell-gtk-and-multi-th...
But substitute Gtk for Qt, or anything where you're the sucker dealing with an external concurrency model or event loop.
Hilariously, they end up having the same debate we are here, just in Haskell.
Seriously, there's no pixie dust for this.
That link is a good example of exactly why we need to push for languages to take concurrency seriously. That's an example of how a haskell user has to deal with the issue when interfacing with a C lib that rolled their own event loop. For most languages people have to deal with that issue all the time, for their own code, entirely in that one language.
I have no idea why you are talking about pixie dust. My point was that explicitly using event loops instead of abstracting it into a green thread library is dumb, and we need to start having higher expectations from languages rather than accepting backwards shit like node. Not "green threads are magic".