The real question is: Why don't you just use Erlang or Elixir?
Can you find the magic sauce in another language?
I believe the honest answer is no. You can't do it in Java because the JVM doesn't support it. You can't really build this on top of something that was not designed with parallelism and concurrency in mind, and designed correctly. For that reason Go doesn't do it either. Scripting languages aren't even thinking about it and to do it in something like C or C++ you'd basically be recreating a poorly implemented half working version of erlang.
So more interesting is--- why do people have this aversion to Erlang and Elixir?
It doesn't take but a couple weeks to get used to erlang programming, and Elixir is even faster I think.
For the record I've been taking interns thru Programming Elixir in about 2 weeks and getting them to where they can program in the language (just at that point now with a new group so we'll see how much they struggle... but I've already seen code written by them.)
Erlang did it right and has put in several decades of effort getting it even more right. Other languages, like Go, aren't even trying (though of course advertising "concurrency" is very popular, none of them really do it.)
All these people choosing things like node.js or go for problems where erlang is the correct solution make me think that too many of the people in our industry aren't really engineers, but more scripters following the herd.
And engineer knows why erlang is the correct solution for distributed systems.
Because you may want good performance without dropping down to C. Many serious Erlang applications are mixed Erlang/C applications, where they delegate processing to C, or use Erlang for the control plain and C for the data plain.
Or because you want more sophisticated, fast transactional shared-state than ETS allows.
Or because you want to take advantage of a much wider selection of high-quality libraries.
Note that these aren't problems with Erlang or Elixir as languages -- they're both great -- but rather with the runtime they're running on, whose developers simply don't have the resources to make it all that it can be. Which brings me to:
the JVM doesn't support it.
The JVM most certainly does support it. It was designed with parallelism and concurrency in mind, and includes some of the best high-quality, production-ready implementations of concurrent data structures and schedulers in existence. The effort that's been poured into the JVM -- and even JVM concurrency alone -- dwarfs the effort put into Erlang at least by an order of magnitude.
I think Erlang would benefit greatly if it were to target the JVM. There is a project, Erjang, that runs BEAM code on the JVM, but it is not very actively maintained and doesn't take advantage of some of the recent, relevant innovations in the Java world. Since that project was developed, there have been major improvements in the pertinent areas on the JVM: concurrent GCs, JITting of dynamically-typed languages, fibers and work-stealing scheduling.
OpenJDK is the second-largest open-source project in the world (after the Linux kernel), and I don't see why Erlang shouldn't take advantage of the vast resources it has at its disposal. OpenJDK and Erlang are a great fit!
My aversion comes from relatively poor language syntax - it's more verbose than it should be. Lack of powerful type system. And less-than-stellar performance. Out of these, the language is the most annoying.
Can you state which language constructs Erlang has that other languages don't, that are critical to concurrency? Cause it looks like someone built a great VM and cluster library, then tossed in a language for fun.
And please explain why Erlang's features are not possible in other languages. Not just unavailable at the moment. Or do you mean the guarantees are different? As in other languages will let you have corrupted shared state? (I view that as a positive; let me escape guarantees when I want too.)
Or perhaps point me to a decent intro that'd cover all this? I looked at it before and didn't get the real point why we need Erlang the language or why in theory, you can't have OTP with the CLR or JVM.
Syntax does not matter much; Erlang's focus is not to be a good general-purpose language. It is good for distributed computing because it lacks many useful and pleasant features.
Erlang has built-in immutability done right (more so than in Scala, OCaml, Clojure(?)) - if it's not built in the language from the start, there's nothing you can do about it.
Erlang has built-in binary serialization/deserialization that makes messages across network easy and natural. Please show me a language that can serialize a closure and send it over network. A rich type system and general-purpose language features are obstacles here, types of values have to be dumb to serialize seamlessly.
Other than that, it has a lot of convenient and coherent features like good pattern-matching (useful for parsing and routing messages), proper tail-call optimization (just look at Scala, again), the built-in actor model, lightweight processes, process monitoring, the large library of battle-tested distributed practices, OTP, – these things can be recreated in another language/VM, but it will probably take huge resources and efforts and self-limitation of the platform, not features.
Comments
The real question is: Why don't you just use Erlang or Elixir?
Can you find the magic sauce in another language?
I believe the honest answer is no. You can't do it in Java because the JVM doesn't support it. You can't really build this on top of something that was not designed with parallelism and concurrency in mind, and designed correctly. For that reason Go doesn't do it either. Scripting languages aren't even thinking about it and to do it in something like C or C++ you'd basically be recreating a poorly implemented half working version of erlang.
So more interesting is--- why do people have this aversion to Erlang and Elixir?
It doesn't take but a couple weeks to get used to erlang programming, and Elixir is even faster I think.
For the record I've been taking interns thru Programming Elixir in about 2 weeks and getting them to where they can program in the language (just at that point now with a new group so we'll see how much they struggle... but I've already seen code written by them.)
Erlang did it right and has put in several decades of effort getting it even more right. Other languages, like Go, aren't even trying (though of course advertising "concurrency" is very popular, none of them really do it.)
All these people choosing things like node.js or go for problems where erlang is the correct solution make me think that too many of the people in our industry aren't really engineers, but more scripters following the herd.
And engineer knows why erlang is the correct solution for distributed systems.
Because you may want good performance without dropping down to C. Many serious Erlang applications are mixed Erlang/C applications, where they delegate processing to C, or use Erlang for the control plain and C for the data plain.
Or because you want more sophisticated, fast transactional shared-state than ETS allows.
Or because you want to take advantage of a much wider selection of high-quality libraries.
Note that these aren't problems with Erlang or Elixir as languages -- they're both great -- but rather with the runtime they're running on, whose developers simply don't have the resources to make it all that it can be. Which brings me to:
The JVM most certainly does support it. It was designed with parallelism and concurrency in mind, and includes some of the best high-quality, production-ready implementations of concurrent data structures and schedulers in existence. The effort that's been poured into the JVM -- and even JVM concurrency alone -- dwarfs the effort put into Erlang at least by an order of magnitude.
I think Erlang would benefit greatly if it were to target the JVM. There is a project, Erjang, that runs BEAM code on the JVM, but it is not very actively maintained and doesn't take advantage of some of the recent, relevant innovations in the Java world. Since that project was developed, there have been major improvements in the pertinent areas on the JVM: concurrent GCs, JITting of dynamically-typed languages, fibers and work-stealing scheduling.
OpenJDK is the second-largest open-source project in the world (after the Linux kernel), and I don't see why Erlang shouldn't take advantage of the vast resources it has at its disposal. OpenJDK and Erlang are a great fit!
My aversion comes from relatively poor language syntax - it's more verbose than it should be. Lack of powerful type system. And less-than-stellar performance. Out of these, the language is the most annoying.
Can you state which language constructs Erlang has that other languages don't, that are critical to concurrency? Cause it looks like someone built a great VM and cluster library, then tossed in a language for fun.
And please explain why Erlang's features are not possible in other languages. Not just unavailable at the moment. Or do you mean the guarantees are different? As in other languages will let you have corrupted shared state? (I view that as a positive; let me escape guarantees when I want too.)
Or perhaps point me to a decent intro that'd cover all this? I looked at it before and didn't get the real point why we need Erlang the language or why in theory, you can't have OTP with the CLR or JVM.
Syntax does not matter much; Erlang's focus is not to be a good general-purpose language. It is good for distributed computing because it lacks many useful and pleasant features.
Erlang has built-in immutability done right (more so than in Scala, OCaml, Clojure(?)) - if it's not built in the language from the start, there's nothing you can do about it.
Erlang has built-in binary serialization/deserialization that makes messages across network easy and natural. Please show me a language that can serialize a closure and send it over network. A rich type system and general-purpose language features are obstacles here, types of values have to be dumb to serialize seamlessly.
Other than that, it has a lot of convenient and coherent features like good pattern-matching (useful for parsing and routing messages), proper tail-call optimization (just look at Scala, again), the built-in actor model, lightweight processes, process monitoring, the large library of battle-tested distributed practices, OTP, – these things can be recreated in another language/VM, but it will probably take huge resources and efforts and self-limitation of the platform, not features.