FWIW, one of the above is widely used in production systems.
and none of the above have solved preserving types across machine boundaries (or at least not in a stable/non-experimental let's-bet-the-bank-on-this release).
Quasar looks pretty insane at first blush; if Java 9 or 10 delivers on the missing piece [1] for this library, watch out.
[1] "Quasar fibers are implemented by creating and scheduling continuation tasks and since the JVM doesn’t (yet) support native continuations, Quasar implements them through selective bytecode instrumentation: methods that can block a fiber currently need to be explicitly marked through annotations so that Quasar can insert the continuation suspension and resumption hooks."
Main author of Quasar here. Quasar's bytecode instrumentation is really very gentle -- no class fields/methods are added/removed and object state isn't changed in any way. It is far, far less intrusive than, say, JRebel. While Java 9 sadly won't include native JVM fibers or continuations, it will probably make the instrumentation process 100% transparent.
Java 9 will introduce the G1 GC as default. How do the emulation of continuations behave regarding this GC? Is is possible to obtain truly pauseless behavior with Quaser and G1?
When you build highly distributed systems, making sure a value is properly transferred across different types of machine is actually not a trivial task because sender and receiver may encode values differently at the bit level (for example, because different endianess system is used). Erlang VM take care of this naturally, and the language offers built in functions to encode / decode binary representations that can be sent over basic transmission channels (like TCP / IP sockets).
If Erlang is not your cup of tea, look at htons() and nstoh() functions to have another example.
My trade is in real time distributed systems applied to the automatic control of trains. You wouldn't believe the number of times I have had to fix this issue in multiple codes, both new or legacy.
And my example is limited to individual values, not data structures. Think about how ugly things can turn if you have things like lists or dictionaries.
When you consider that Erlang adds on top of this built-in marshalling and demarshalling of data structure. As of today, I haven't found a more efficent environment to do this kind of stuff safely.
How's that different than using a well defined binary serialization format (protobufs?) and having a controller daemon that makes sure everyone reports the same version or has the same serialization contracts loaded before adding them to discovery?
Or is it just that Erlang has this all batteries included, the way Python's popular with scientific programming?
You are spot on. You can do what Erlang does in many ways. It's the combination of all this, built into a coherent architecture, that makes OTP unique.
The reason this thread exists is because other options with strong ecosystems and better performance which accomplish the above mentioned stuff just don't seem to exist.
I'd actually love to see some performance numbers comparing performance on Erlang/OTP vs the alternatives, though. I can't find anything quite like that, maybe someone knows a link.
Similar solutions exist everywhere, which is partly why the Erlang ecosystem is so small!
But I agree that Erlang's design is beautiful, which is why I've decided to port it to the JVM with Quasar. I wanted to bring that great design to a more powerful runtime and a much bigger ecosystem. Again, the problem isn't with the Erlang language but with the runtime that, while really nice, simply does not have nearly enough resources to compete with the bigger runtimes like the JVM and .NET (or even V8).
We will have some benchmarks eventually, I promise, but those take a long time and aren't too critical, because, like I said, the Erlang ecosystem is very small.
P.S., in case that wasn't clear: Erlang's design is absolutely brilliant -- it's inspired -- but the implementation suffers from insufficient resources. The result is lagging performance and a small ecosystem.
Akka production release is all about `Any`, as in it's untyped, and you have to pattern match on the receiver to the expected type.
That's a far cry from Akka Typed [1], the recently released experimental branch, which allows you to match on the "real" type; as a result distributed code is compile time checked...a dream scenario ;-)
and none of the above have solved preserving types across machine boundaries (or at least not in a stable/non-experimental let's-bet-the-bank-on-this release).
What about Static Pointers[0]? Do you feel the solve the problem or are likely to if you characterize them as non-stable and/or experimental?
Comments
FWIW, one of the above is widely used in production systems.
and none of the above have solved preserving types across machine boundaries (or at least not in a stable/non-experimental let's-bet-the-bank-on-this release).
Quasar looks pretty insane at first blush; if Java 9 or 10 delivers on the missing piece [1] for this library, watch out.
[1] "Quasar fibers are implemented by creating and scheduling continuation tasks and since the JVM doesn’t (yet) support native continuations, Quasar implements them through selective bytecode instrumentation: methods that can block a fiber currently need to be explicitly marked through annotations so that Quasar can insert the continuation suspension and resumption hooks."
Main author of Quasar here. Quasar's bytecode instrumentation is really very gentle -- no class fields/methods are added/removed and object state isn't changed in any way. It is far, far less intrusive than, say, JRebel. While Java 9 sadly won't include native JVM fibers or continuations, it will probably make the instrumentation process 100% transparent.
Java 9 will introduce the G1 GC as default. How do the emulation of continuations behave regarding this GC? Is is possible to obtain truly pauseless behavior with Quaser and G1?
I'm not familiar with Erlang: what does preserving types across machine boundaries mean in this context?
When you build highly distributed systems, making sure a value is properly transferred across different types of machine is actually not a trivial task because sender and receiver may encode values differently at the bit level (for example, because different endianess system is used). Erlang VM take care of this naturally, and the language offers built in functions to encode / decode binary representations that can be sent over basic transmission channels (like TCP / IP sockets).
If Erlang is not your cup of tea, look at htons() and nstoh() functions to have another example.
My trade is in real time distributed systems applied to the automatic control of trains. You wouldn't believe the number of times I have had to fix this issue in multiple codes, both new or legacy.
And my example is limited to individual values, not data structures. Think about how ugly things can turn if you have things like lists or dictionaries.
When you consider that Erlang adds on top of this built-in marshalling and demarshalling of data structure. As of today, I haven't found a more efficent environment to do this kind of stuff safely.
How's that different than using a well defined binary serialization format (protobufs?) and having a controller daemon that makes sure everyone reports the same version or has the same serialization contracts loaded before adding them to discovery?
Or is it just that Erlang has this all batteries included, the way Python's popular with scientific programming?
You are spot on. You can do what Erlang does in many ways. It's the combination of all this, built into a coherent architecture, that makes OTP unique.
You call it coherent, I call it a small ecosystem with limited choices, limited development resources behind it, and disappointing performance.
The reason this thread exists is because other options with strong ecosystems and better performance which accomplish the above mentioned stuff just don't seem to exist.
I'd actually love to see some performance numbers comparing performance on Erlang/OTP vs the alternatives, though. I can't find anything quite like that, maybe someone knows a link.
Similar solutions exist everywhere, which is partly why the Erlang ecosystem is so small!
But I agree that Erlang's design is beautiful, which is why I've decided to port it to the JVM with Quasar. I wanted to bring that great design to a more powerful runtime and a much bigger ecosystem. Again, the problem isn't with the Erlang language but with the runtime that, while really nice, simply does not have nearly enough resources to compete with the bigger runtimes like the JVM and .NET (or even V8).
We will have some benchmarks eventually, I promise, but those take a long time and aren't too critical, because, like I said, the Erlang ecosystem is very small.
P.S., in case that wasn't clear: Erlang's design is absolutely brilliant -- it's inspired -- but the implementation suffers from insufficient resources. The result is lagging performance and a small ecosystem.
Probably referring to the external term format used in multi-node distributed message passing: http://erlang.org/doc/apps/erts/erl_ext_dist.html
Akka preserves types across machine boundaries, at least inasmuch as the External Term Format does, AFAIK.
Would you say that it's missing something?
Akka production release is all about `Any`, as in it's untyped, and you have to pattern match on the receiver to the expected type.
That's a far cry from Akka Typed [1], the recently released experimental branch, which allows you to match on the "real" type; as a result distributed code is compile time checked...a dream scenario ;-)
[1] http://doc.akka.io/docs/akka/snapshot/scala/typed.html
Yup, I've been using Akka Typed myself recently.
Having said that it doesn't change the fact that Akka preserves types in most arbitrary hierarchies today, across machine boundaries.
What about Static Pointers[0]? Do you feel the solve the problem or are likely to if you characterize them as non-stable and/or experimental?
0: https://ocharles.org.uk/blog/guest-posts/2014-12-23-static-p...