I think cassandra is pretty good for analytical workloads...for instance "Give me all the customers from pittsburgh" is a faster scan when the city column is laid out contiguously on disk.
Cassandra doesn't store columns from different rows together to my knowledge. That means it is no way like an analytic column store where columns from all rows are stored together. This is not a bad thing, it just wasn't intended to be an analytic column store and makes design choices suited for other things.
Analytic column stores can exploit storing columns separately by using various encoding schemes to compress data that are far superior to generalized compression and they can use the sort to zero on relevant ranges for each column.
Ah ok. I understand why a columnar database would be better for analytical workloads, but I didn't realize that HBase and Cassandra aren't actually laid out in a columnar fashion on disk. I was confused by the 'column store' terminology. This makes Citus data's release even more compelling.
An analytic column store like say Vertica has a schema like a regular SQL database. I don't know what their flexible schema story is right now.
Instead of storing the columns of a row together an analytic column store will store columns from many rows together in sorted runs. When you go to do a scan your disk will only read the columns you have selected. The format for column storage is optimized for specific types and uses type specific compression so 10-50x is something that is claimed. This further improves the IO situation. They can also zero in on relevant ranges of data for each column because they are indexed and this further reduces the IO requirements.
Where other databases are bound on seeks or sequential throughput an analytic column store will be bound on CPU, especially CPU for the non-parallel portions of every query.
Obviously a column store will have a hard time selecting individual rows because the data is not stored together so it will be expensive to materialize. They also have trouble with updates/deletes to already inserted data, in some cases requiring the data be reloaded because updates have dragged everything down.
Comments
Cassandra is a Bigtable style column store not an analytic column store.
It's an unfortunate naming collision.
I think cassandra is pretty good for analytical workloads...for instance "Give me all the customers from pittsburgh" is a faster scan when the city column is laid out contiguously on disk.
Cassandra doesn't store columns from different rows together to my knowledge. That means it is no way like an analytic column store where columns from all rows are stored together. This is not a bad thing, it just wasn't intended to be an analytic column store and makes design choices suited for other things.
See http://wiki.apache.org/cassandra/ArchitectureSSTable
Analytic column stores can exploit storing columns separately by using various encoding schemes to compress data that are far superior to generalized compression and they can use the sort to zero on relevant ranges for each column.
Ah ok. I understand why a columnar database would be better for analytical workloads, but I didn't realize that HBase and Cassandra aren't actually laid out in a columnar fashion on disk. I was confused by the 'column store' terminology. This makes Citus data's release even more compelling.
Thanks for clearing that up!
can you kindly provide at a high level the differences between the two?
For Cassandra see http://www.datastax.com/docs/0.8/ddl/index
An analytic column store like say Vertica has a schema like a regular SQL database. I don't know what their flexible schema story is right now.
Instead of storing the columns of a row together an analytic column store will store columns from many rows together in sorted runs. When you go to do a scan your disk will only read the columns you have selected. The format for column storage is optimized for specific types and uses type specific compression so 10-50x is something that is claimed. This further improves the IO situation. They can also zero in on relevant ranges of data for each column because they are indexed and this further reduces the IO requirements.
Where other databases are bound on seeks or sequential throughput an analytic column store will be bound on CPU, especially CPU for the non-parallel portions of every query.
Obviously a column store will have a hard time selecting individual rows because the data is not stored together so it will be expensive to materialize. They also have trouble with updates/deletes to already inserted data, in some cases requiring the data be reloaded because updates have dragged everything down.