Yes, this is necessary. Now, build up a few more of these patterns, start watching them intertwine with each other in ways that require you to manually interleave them, come to the realization that building your glorious asynchronous stack on top of a fundamentally synchronous language was a bad idea.
In languages that are not fundamentally synchronous, such as Erlang, you do not leap through hoops to manage this. You simply write a function that performs your insertions in the straightforward and obvious way, and the runtime manages it with no blocking at any point.
I actually don't like the frequently-made assertion that "a pattern is automatically a weakness in the language", but it does apply here. As the Node.js community laboriously builds up the patterns necessary to work in this paradigm, recapitulating the work done in numerous other async-on-top-of-sync-language libraries, it probably is worth keeping the assertion in mind. You shouldn't even have to think about this, let alone argue which way is the best way to do it.
(Somebody modded this down. Maybe I should make it clear that I'm actually speaking from experience on another asynchronous project written in Perl on top of glib's asynchronousness, which isn't fundamentally different from Node.js'. I'm not speculating, I've been on the receiving end of this complexity explosion. It will happen.)
I've been on the receiving end of this complexity explosion
Count me in. Been there with Twisted and (partly) EventMachine.
Imho node should really abstract this away before the eco-system turns into a giant spaghetti ball. Of course you can't bend javascript into a concurrency model as elegant as erlang's. But co-routines or a similar abstraction is urgently needed at least for the code that the users end up writing.
On a related note; I recently switched to coffeescript in an effort to bring my (still small) node codebase back into a half-way readable state.
The simple act of removing most curly brackets had an astonishing impact on readability. I still have to reason about callback chains, but at least I no more have to wade through half screens full of closing brackets while doings so.
That observation was a bitter-sweet reminder of what I'm (willfully) getting myself into with node - all the while keeping my fingers crossed that this wart on an otherwise beautiful environment will be improved asap.
A long time ago I convinced or at least started the ball rolling on getting coroutines added to node.js. Now when they finally were added, it was done in what I would call a very wrong fashion. Coroutines could have events pushed to them without being yielded to (there needs to be a guard). Of course the reentry issue is just as bad as the one being solved so I think it was written off as a horrible idea and never returned to.
Of course, it didn't have to be that way if they had proper event triggering semantics. I find very little hope of the community returning to this issue though. Not that it's the only way to solve things... but it would have been a nice balance IMO. (and I still hold that coroutines can be done cheaply... at least as cheaply as event sources if not more)
(Update: to clarify on proper coroutines on node, each needs to be able to iterate on an independent tick loop rather than allow preemption from EventEmitter#emit)
I completely agree here. While I think node.js will have a certain amount of success, its biggest limit is and will continue to be the utterly poor composability of code. You can't just take other code and call it anymore. It has to be written to interact not only asynchronously but ALSO track and maintain state the same way other code is handled that might be using it.
Solutions that make this transparent almost always return to heavy constructs that people like Ryan Dahl called out as big scaling problems (like threads). Just shoveling state around is a mess and abstractions around it almost always reinvent the smalltalk-style spaghetti stack.
I'm not sure node.js has a clear option moving forward. Right now the library choice for this is pretty wide, so fragmentation is a problem. While it's hard to choose sometimes, right now the community really needs some direction.
Meanwhile, languages and platforms that have already chosen their concurrency primitives, be it goroutines and channels, erlang processes and messages, or even ugly threads and locks, seem to be moving forward. It's not about defeating some other method of expressing things. It's about providing something that works and right now node.js only solves half of the problem.
Comments
Yes, this is necessary. Now, build up a few more of these patterns, start watching them intertwine with each other in ways that require you to manually interleave them, come to the realization that building your glorious asynchronous stack on top of a fundamentally synchronous language was a bad idea.
In languages that are not fundamentally synchronous, such as Erlang, you do not leap through hoops to manage this. You simply write a function that performs your insertions in the straightforward and obvious way, and the runtime manages it with no blocking at any point.
I actually don't like the frequently-made assertion that "a pattern is automatically a weakness in the language", but it does apply here. As the Node.js community laboriously builds up the patterns necessary to work in this paradigm, recapitulating the work done in numerous other async-on-top-of-sync-language libraries, it probably is worth keeping the assertion in mind. You shouldn't even have to think about this, let alone argue which way is the best way to do it.
(Somebody modded this down. Maybe I should make it clear that I'm actually speaking from experience on another asynchronous project written in Perl on top of glib's asynchronousness, which isn't fundamentally different from Node.js'. I'm not speculating, I've been on the receiving end of this complexity explosion. It will happen.)
I've been on the receiving end of this complexity explosion
Count me in. Been there with Twisted and (partly) EventMachine.
Imho node should really abstract this away before the eco-system turns into a giant spaghetti ball. Of course you can't bend javascript into a concurrency model as elegant as erlang's. But co-routines or a similar abstraction is urgently needed at least for the code that the users end up writing.
On a related note; I recently switched to coffeescript in an effort to bring my (still small) node codebase back into a half-way readable state.
The simple act of removing most curly brackets had an astonishing impact on readability. I still have to reason about callback chains, but at least I no more have to wade through half screens full of closing brackets while doings so.
That observation was a bitter-sweet reminder of what I'm (willfully) getting myself into with node - all the while keeping my fingers crossed that this wart on an otherwise beautiful environment will be improved asap.
A long time ago I convinced or at least started the ball rolling on getting coroutines added to node.js. Now when they finally were added, it was done in what I would call a very wrong fashion. Coroutines could have events pushed to them without being yielded to (there needs to be a guard). Of course the reentry issue is just as bad as the one being solved so I think it was written off as a horrible idea and never returned to.
Of course, it didn't have to be that way if they had proper event triggering semantics. I find very little hope of the community returning to this issue though. Not that it's the only way to solve things... but it would have been a nice balance IMO. (and I still hold that coroutines can be done cheaply... at least as cheaply as event sources if not more)
(Update: to clarify on proper coroutines on node, each needs to be able to iterate on an independent tick loop rather than allow preemption from EventEmitter#emit)
I completely agree here. While I think node.js will have a certain amount of success, its biggest limit is and will continue to be the utterly poor composability of code. You can't just take other code and call it anymore. It has to be written to interact not only asynchronously but ALSO track and maintain state the same way other code is handled that might be using it.
Solutions that make this transparent almost always return to heavy constructs that people like Ryan Dahl called out as big scaling problems (like threads). Just shoveling state around is a mess and abstractions around it almost always reinvent the smalltalk-style spaghetti stack.
I'm not sure node.js has a clear option moving forward. Right now the library choice for this is pretty wide, so fragmentation is a problem. While it's hard to choose sometimes, right now the community really needs some direction.
Meanwhile, languages and platforms that have already chosen their concurrency primitives, be it goroutines and channels, erlang processes and messages, or even ugly threads and locks, seem to be moving forward. It's not about defeating some other method of expressing things. It's about providing something that works and right now node.js only solves half of the problem.