I have a naive question - why would I not use something like ZeroMQ for messaging ?
I completely understand the need for JSON when you are communicating with the frontend/Javascript. But if you are doing backend messaging, would you not rather use ZeroMQ (which comes with its own protocol).
From what I understand (from previous HN posts), it is super fast and handles binary data very efficiently. I also understand that you can tune stuff at the OS level to wring the last bit of performance from ZeroMQ.
P.S: yup, it is written in C, but all bindings apparently work very well.
They're different types of thing. ZeroMQ doesn't replace JSON, it replaces HTTP. If you're using ZeroMQ it will help you get some bytes from one place to another, but you still need a serialization format. You might well send JSON over ZeroMQ.
Here's my specific doubt - my application uses arrays, maps, lists, etc. Can I not use zeromq to send these data-structures as binary data rather than serializing them to JSON ?
You're going to have to send them as a sequence of bytes somehow. MessagePack converts your data structures into a straightforward binary format, or lets you roll your own custom serializers easily (I've used it, and it's good stuff), and JSON does something similar in a less compact but more human-readable form. If your data structures are very simple, you could roll your own, e.g. send an array of ints by just converting them to network byte order and sending them.
Once you've got your data expressed as a sequence of bytes, you can send that with ZeroMQ. Or with HTTP, or raw sockets, or whatever. The serialization format and the transport protocol are pretty much completely independent.
Comments
I have a naive question - why would I not use something like ZeroMQ for messaging ?
I completely understand the need for JSON when you are communicating with the frontend/Javascript. But if you are doing backend messaging, would you not rather use ZeroMQ (which comes with its own protocol).
From what I understand (from previous HN posts), it is super fast and handles binary data very efficiently. I also understand that you can tune stuff at the OS level to wring the last bit of performance from ZeroMQ.
P.S: yup, it is written in C, but all bindings apparently work very well.
They're different types of thing. ZeroMQ doesn't replace JSON, it replaces HTTP. If you're using ZeroMQ it will help you get some bytes from one place to another, but you still need a serialization format. You might well send JSON over ZeroMQ.
Here's my specific doubt - my application uses arrays, maps, lists, etc. Can I not use zeromq to send these data-structures as binary data rather than serializing them to JSON ?
You're going to have to send them as a sequence of bytes somehow. MessagePack converts your data structures into a straightforward binary format, or lets you roll your own custom serializers easily (I've used it, and it's good stuff), and JSON does something similar in a less compact but more human-readable form. If your data structures are very simple, you could roll your own, e.g. send an array of ints by just converting them to network byte order and sending them.
Once you've got your data expressed as a sequence of bytes, you can send that with ZeroMQ. Or with HTTP, or raw sockets, or whatever. The serialization format and the transport protocol are pretty much completely independent.