You are likely doing something wrong. My in-memory redis server clone that also uses netty reaches 1.5M requests/s on an arguably heavier workload with a more complicated protocol. I'd love to help you optimize it if it matters.
Thanks for the offer! I, uh, certainly didn't mean to claim that Riemann is extraordinarily fast. Up until this point I've been working as hard as I can just to make the darn thing turn on and accomplish something useful. This is quite early in the optimization process, haha. :)
The principle bottleneck in this particular test is actually the client--Riemann itself spends ~94% of its time waiting on epoll in this test. The client is a total hack using Java OIO, calling flush() on every message, Nagle's algorithm disabled for low per-msg latencies... it's a wreck, haha. Part of next week's optimization push is replacing it with a Netty client and tuning TCP options for various workloads.
Things will probably look better once you switch your client implementation to Netty as a similar test I did with Netty 4, Protobufs and batching of messages produced low seven figure numbers too.
Comments
You are likely doing something wrong. My in-memory redis server clone that also uses netty reaches 1.5M requests/s on an arguably heavier workload with a more complicated protocol. I'd love to help you optimize it if it matters.
Thanks for the offer! I, uh, certainly didn't mean to claim that Riemann is extraordinarily fast. Up until this point I've been working as hard as I can just to make the darn thing turn on and accomplish something useful. This is quite early in the optimization process, haha. :)
The principle bottleneck in this particular test is actually the client--Riemann itself spends ~94% of its time waiting on epoll in this test. The client is a total hack using Java OIO, calling flush() on every message, Nagle's algorithm disabled for low per-msg latencies... it's a wreck, haha. Part of next week's optimization push is replacing it with a Netty client and tuning TCP options for various workloads.
Things will probably look better once you switch your client implementation to Netty as a similar test I did with Netty 4, Protobufs and batching of messages produced low seven figure numbers too.
Ah ok, my test was run using the C redis-benchmark client.
Is the source available anywhere?
1.5M locally or over a network? What is the size of each message?
http://github.com/spullara/redis-protocol — server module
The test was run with "SET key value" which in the redis protocol is something like 15-20 bytes / message.