On a more serious note, looking forward to trying this new release out. I found previous versions really straightforward to get up and running with on Kubernetes but performance was always lacklustre.
Is that saying that you are comparing Cockroach running on 81 c5d.9xls vs Aurora running on 2 r3.8xls? I get that part of what you are trying to show is that Cockroach will scale far past what any single-master system can, but it feels pretty lame to run a test comparing transaction throughput on 2900 cores vs 64 cores and a data set size comparison on 81 hosts vs 2.
The sysbench metrics seems like a much fairer comparison, and CockroachDB looks great in those metrics as well, so I don't really get why you are leading with a comparison that looks really sketchy at first glance.
Hi- product manager from CRL here. You are right--we hope to demonstrate that CockroachDB is built to scale horizontally. Deployments of CockroachDB can grow by easily adding more nodes to the cluster which in turn linearly scales throughput. We have posted the most recent published Aurora numbers as a comparison to demonstrate how architecture can influence scale.
We also hear your point about efficiency as tpmC (throughput) alone isn't sufficient to compare systems without taking hardware into account. TPC-C asks users to provide a tpmC per dollar amount. We conducted this price comparison previously in this blog post https://www.cockroachlabs.com/blog/cockroachdb-2dot1-perform.... These results are even lower in 19.2 because we can achieve greater tpmC with fewer nodes.
This page wasn't met to be competitive-we simply showed Aurora as a reference point. Since we want to focus on CockroachDB we will remove the Aurora comparison.
Others have commented about how the language that CockroachDB was written in causes performance issues due to Go using a GC. Can't say that this is the main reason for sure, but I guess it could be a possibility.
Comments
Waiting for the comments about the name.
On a more serious note, looking forward to trying this new release out. I found previous versions really straightforward to get up and running with on Kubernetes but performance was always lacklustre.
I respect the name at this point. It won't go away - just like a real Cockroach and a distributed fault tolerant database system.
You can find our more recent performance numbers on https://www.cockroachlabs.com/docs/v19.2/performance.html
Is that saying that you are comparing Cockroach running on 81 c5d.9xls vs Aurora running on 2 r3.8xls? I get that part of what you are trying to show is that Cockroach will scale far past what any single-master system can, but it feels pretty lame to run a test comparing transaction throughput on 2900 cores vs 64 cores and a data set size comparison on 81 hosts vs 2.
The sysbench metrics seems like a much fairer comparison, and CockroachDB looks great in those metrics as well, so I don't really get why you are leading with a comparison that looks really sketchy at first glance.
Hi- product manager from CRL here. You are right--we hope to demonstrate that CockroachDB is built to scale horizontally. Deployments of CockroachDB can grow by easily adding more nodes to the cluster which in turn linearly scales throughput. We have posted the most recent published Aurora numbers as a comparison to demonstrate how architecture can influence scale.
We also hear your point about efficiency as tpmC (throughput) alone isn't sufficient to compare systems without taking hardware into account. TPC-C asks users to provide a tpmC per dollar amount. We conducted this price comparison previously in this blog post https://www.cockroachlabs.com/blog/cockroachdb-2dot1-perform.... These results are even lower in 19.2 because we can achieve greater tpmC with fewer nodes.
This page wasn't met to be competitive-we simply showed Aurora as a reference point. Since we want to focus on CockroachDB we will remove the Aurora comparison.
Performance in what sense? Searching online for benchmarks gives some impressive numbers, specifically this: https://www.cockroachlabs.com/blog/cockroachdb-2dot1-perform....
Others have commented about how the language that CockroachDB was written in causes performance issues due to Go using a GC. Can't say that this is the main reason for sure, but I guess it could be a possibility.
Try it out on your stack and with your data and see.