Very interesting. We're currently working on a quick project to test integrating Hazelcast into a distributed server to use as a queue. Did you look at it by any chance? Curious if it was missing something that you needed.
From my personal experience with these, Hazelcast seems to be better suited for something like a single worker queue... We had issues when we had thousands of distributed queues on different systems. We ended up going with Terracotta (which handles this well if you work through the lock contention).
This project sounds promising for that kind of application but wont know until I try it out.
Sorry for my confusion, so is the problem you experienced one of having thousands of queues that a small number of clients pulled from, or thousands of workers that worked off of a small number of queues?
In our case, we want to be having a very small number of queues that have a high rate of i/o with dozens of reader clients and writer clients.
Could you eloborate where you did have problems?
We also plan on using a small number of queues having a high rate of readers and clients (e.g. distributed task execution).
Comments
Very interesting. We're currently working on a quick project to test integrating Hazelcast into a distributed server to use as a queue. Did you look at it by any chance? Curious if it was missing something that you needed.
From my personal experience with these, Hazelcast seems to be better suited for something like a single worker queue... We had issues when we had thousands of distributed queues on different systems. We ended up going with Terracotta (which handles this well if you work through the lock contention).
This project sounds promising for that kind of application but wont know until I try it out.
Sorry for my confusion, so is the problem you experienced one of having thousands of queues that a small number of clients pulled from, or thousands of workers that worked off of a small number of queues?
In our case, we want to be having a very small number of queues that have a high rate of i/o with dozens of reader clients and writer clients.
Could you eloborate where you did have problems? We also plan on using a small number of queues having a high rate of readers and clients (e.g. distributed task execution).
We have a large number of queues with small number of workers.