We've been using Citus at MixRank for storing our timeseries data, and it's worked out magnificently well for our use-case.
A few points:
(i) We can do ad-hoc realtime analytics on hundreds of millions of data points.
(ii) We can also do realtime analytics on billions of datapoints as long as we pre-compute along one dimension.
(iii) We could do a lot better at (i) and (ii) if we invested more heavily in hardware (and Citus would make this pretty painless, actually).
(iv) I'd normally not consider a closed-source solution personally, but since Citus is based so heavily on PostgreSQL (protocol-level compatibility, configuration, codebase), this has been a non-issue for us. We can still lean on the amazing PostgreSQL community, documentation, and for the parts we don't have the source code to, the Citus team has been very helpful in explaining how things work.
(v) Fault tolerance is immaculate. At the node level, PostgreSQL is notoriously one of the most reliable and robust databases available. At the cluster level, Citus will magically fall back to a replica mid-query when a server dies.
(vi) Although realtime inserts are not supported out of the box, the system is flexible enough that we were able to get this working on our own without help from Citus.
(vii) Schema migrations are also not supported out of the box, but we built a schema migration framework that takes care of this for us.
(viii) We're not worried about vendor lock-in, since the data is just stored on our servers, in the PostgreSQL serialization format. If we wanted to, we could just give up the features that Citus gives us and build our own data-access layer on top of our cluster.
Anyway, it won't be everything to everyone, but it works very well for our OLAP use-case of timeseries ad impression data. I'd definitely recommend looking into it if you're otherwise considering Hadoop, Vertica, Aster, Greenplum, or a sharded MySQL/PostgreSQL setup.
Full disclosure: I am extremely biased since I've gotten to know the team very well after using Citus. I'm definitely one of their biggest fans, if for no other reason the amount of time they've saved us at MixRank.
Hi Smilliken
Would it be possible to elaborate a bit on how you do real time inserts/updates?
I'm interested in trying Citus but would most probably need the realtime feature for production use
Comments
We've been using Citus at MixRank for storing our timeseries data, and it's worked out magnificently well for our use-case.
A few points:
(i) We can do ad-hoc realtime analytics on hundreds of millions of data points.
(ii) We can also do realtime analytics on billions of datapoints as long as we pre-compute along one dimension.
(iii) We could do a lot better at (i) and (ii) if we invested more heavily in hardware (and Citus would make this pretty painless, actually).
(iv) I'd normally not consider a closed-source solution personally, but since Citus is based so heavily on PostgreSQL (protocol-level compatibility, configuration, codebase), this has been a non-issue for us. We can still lean on the amazing PostgreSQL community, documentation, and for the parts we don't have the source code to, the Citus team has been very helpful in explaining how things work.
(v) Fault tolerance is immaculate. At the node level, PostgreSQL is notoriously one of the most reliable and robust databases available. At the cluster level, Citus will magically fall back to a replica mid-query when a server dies.
(vi) Although realtime inserts are not supported out of the box, the system is flexible enough that we were able to get this working on our own without help from Citus.
(vii) Schema migrations are also not supported out of the box, but we built a schema migration framework that takes care of this for us.
(viii) We're not worried about vendor lock-in, since the data is just stored on our servers, in the PostgreSQL serialization format. If we wanted to, we could just give up the features that Citus gives us and build our own data-access layer on top of our cluster.
Anyway, it won't be everything to everyone, but it works very well for our OLAP use-case of timeseries ad impression data. I'd definitely recommend looking into it if you're otherwise considering Hadoop, Vertica, Aster, Greenplum, or a sharded MySQL/PostgreSQL setup.
Full disclosure: I am extremely biased since I've gotten to know the team very well after using Citus. I'm definitely one of their biggest fans, if for no other reason the amount of time they've saved us at MixRank.
Hi Smilliken Would it be possible to elaborate a bit on how you do real time inserts/updates? I'm interested in trying Citus but would most probably need the realtime feature for production use
I'd be happy to help, but this is probably out of scope of an HN comment. Feel free to reach out to the email in my profile.