The CPU is rarely the bottleneck and for both Redis/HAProxy the vertical scalability solution has been to launch multiple processes or forks with different core affinities. There are downsides of course (no IPC) but I still argue that CPU is not the bottleneck for 99% of usage scenarios.
HAProxy added threading support in 1.8 as you pointed out and Redis has started the same (for a certain subset of processing) in 4.0 as well. They're getting there but concurrency is tough.
To suggest that his product is a "dead end" due to not supporting threading seems a bit premature, as Redis and HAProxy are extremely well-regarded in their niche and they made it there without threading, and we've been at maximal clock speed for nearly a decade.
There are downsides of course (no IPC) but I still argue that CPU is not the bottleneck for 99% of usage scenarios.
I suppose my experience might be unusual, but I frequently log in to c3.8xlarge redis machines that have a single core pegged at 100% and the rest doing nothing. Yes multiple processes help, but that requires updating clients and makes it harder to share memory.
To suggest that his product is a "dead end" due to not supporting threading seems a bit premature, as Redis and HAProxy are extremely well-regarded in their niche and they made it there without threading.
Well yeah, CPUs hitting their GHZ limit and the dramatic increase in the number of cores per machine is a relatively recent phenomena.
I just think its weird to start a brand new project making those same assumptions, especially when the underlying programming language was explicitly designed with concurrency in mind.
It'd be like building a new networking library in Rust which ditches memory safety.
Comments
The CPU is rarely the bottleneck and for both Redis/HAProxy the vertical scalability solution has been to launch multiple processes or forks with different core affinities. There are downsides of course (no IPC) but I still argue that CPU is not the bottleneck for 99% of usage scenarios.
HAProxy added threading support in 1.8 as you pointed out and Redis has started the same (for a certain subset of processing) in 4.0 as well. They're getting there but concurrency is tough.
To suggest that his product is a "dead end" due to not supporting threading seems a bit premature, as Redis and HAProxy are extremely well-regarded in their niche and they made it there without threading, and we've been at maximal clock speed for nearly a decade.
I suppose my experience might be unusual, but I frequently log in to c3.8xlarge redis machines that have a single core pegged at 100% and the rest doing nothing. Yes multiple processes help, but that requires updating clients and makes it harder to share memory.
Well yeah, CPUs hitting their GHZ limit and the dramatic increase in the number of cores per machine is a relatively recent phenomena.
I just think its weird to start a brand new project making those same assumptions, especially when the underlying programming language was explicitly designed with concurrency in mind.
It'd be like building a new networking library in Rust which ditches memory safety.