If I had any feedback, it might be to produce a frank document of trade-offs in your system: workloads that work very well, pathological workloads that perform poorly or lead to some awkward code, given the mechanisms at play. I'm sure you have chosen your trade-offs carefully, or else you wouldn't have bothered doing the hard work to commercialize them. Here you may want to assess the advantages of other general approaches (I think specific implementations would be too much detail and invite a quagmire of nit-picking) versus your own, but the problems they also create that you do not have.
It's something I'm going to try to do anyway, but it would make me feel as though I have a more constructive relationship with the company rather than trying to guess what mix of design decisions you fall in, probably all of which have at least been implemented at least once independently before in some system or another -- however, your mix is likely novel.
However, I don't see this done very often in proprietary offerings (that's still the case for FoundationDB, right?), and I'm not sure if that's because such frankness is a stupid idea or if because everyone is just doing what everyone around them does. It is something that more collaborative open source projects do more freely.
We'd be happy to give you information that would be helpful for your document - your offer to put something like that together is generous, helpful, and we'll definitely take you up on it.
Why don't you shoot me an email at nick dot lavezzo at foundationdb. If you're at Disrupt or in the SF area we might even be able to get together before we head back to DC on Thursday morning.
Thanks again for the helpful comments and interest!
I am speaking more generally -- I'm not evaluating a new database for my toolchain at the moment, and to be honest, I am very shy of anything that is proprietary-only. However, I thought your ideas were interesting regardless, so I felt compelled to defend their interesting-ness.
I'm just tired of trying pierce through proprietary database literature language to sort a new database system that has Apparently Magical Properties into the multi-dimensional bin of tradeoffs. Consider the FAQ for Redis: http://redis.io/topics/faq
Or, the very short rundown in the second half of the home page of Postgres-XC http://postgres-xc.sourceforge.net/, although it probably could use some fine-tuning. Many of the developers, including the first one, are not native english speakers.
How does Datomic handle consistency, availability and scale? Because
Datomic separates the handling of transaction processing and
query/read, it can make independent choices for each
facility. Transactions favor consistency over availability, and the
loss of a transactor will require traditional failover
approaches. Queries and reads, however, are serviced by the highly
available, replicated and scalable infrastructure of DynamoDB. And,
since the data segments stored by Datomic in Dynamo are immutable,
they are always consistent. Thus, Datomic is well suited for
applications that require write consistency and read scalability.
On the other hand, I really appreciate your benchmarks to give me a ballpark idea, and think it's great that you offer them up so easily. They could probably take a less pronounced position in your literature if you find you are in need of space for other material, though.
I see what you mean - thanks for the clarification. We'll be working on a FAQ to get up on the website soon, and we'll also be putting up a blog where we'll cover areas of particular interest in detail. I'm sure CAP will be one of the earliest topics covered, but we've made a lot of important design decisions while building this product and they should all make for interesting reading.
We've done our best to take the technical high road in those hard decisions - generally trading in ease of (our) development for stronger guarantees, more rigorous testing, and higher performance. We're excited to talk about our development process and decisions with the world. Looks like we've got some writing to do :)
Comments
If I had any feedback, it might be to produce a frank document of trade-offs in your system: workloads that work very well, pathological workloads that perform poorly or lead to some awkward code, given the mechanisms at play. I'm sure you have chosen your trade-offs carefully, or else you wouldn't have bothered doing the hard work to commercialize them. Here you may want to assess the advantages of other general approaches (I think specific implementations would be too much detail and invite a quagmire of nit-picking) versus your own, but the problems they also create that you do not have.
It's something I'm going to try to do anyway, but it would make me feel as though I have a more constructive relationship with the company rather than trying to guess what mix of design decisions you fall in, probably all of which have at least been implemented at least once independently before in some system or another -- however, your mix is likely novel.
However, I don't see this done very often in proprietary offerings (that's still the case for FoundationDB, right?), and I'm not sure if that's because such frankness is a stupid idea or if because everyone is just doing what everyone around them does. It is something that more collaborative open source projects do more freely.
Different FoundationDB founder here :)
We'd be happy to give you information that would be helpful for your document - your offer to put something like that together is generous, helpful, and we'll definitely take you up on it.
Why don't you shoot me an email at nick dot lavezzo at foundationdb. If you're at Disrupt or in the SF area we might even be able to get together before we head back to DC on Thursday morning.
Thanks again for the helpful comments and interest!
I am speaking more generally -- I'm not evaluating a new database for my toolchain at the moment, and to be honest, I am very shy of anything that is proprietary-only. However, I thought your ideas were interesting regardless, so I felt compelled to defend their interesting-ness.
I'm just tired of trying pierce through proprietary database literature language to sort a new database system that has Apparently Magical Properties into the multi-dimensional bin of tradeoffs. Consider the FAQ for Redis: http://redis.io/topics/faq
Or, the very short rundown in the second half of the home page of Postgres-XC http://postgres-xc.sourceforge.net/, although it probably could use some fine-tuning. Many of the developers, including the first one, are not native english speakers.
A good example in proprietary-land is Datomic: http://www.datomic.com/faq.html. Here's a good paragraph:
On the other hand, I really appreciate your benchmarks to give me a ballpark idea, and think it's great that you offer them up so easily. They could probably take a less pronounced position in your literature if you find you are in need of space for other material, though.I see what you mean - thanks for the clarification. We'll be working on a FAQ to get up on the website soon, and we'll also be putting up a blog where we'll cover areas of particular interest in detail. I'm sure CAP will be one of the earliest topics covered, but we've made a lot of important design decisions while building this product and they should all make for interesting reading.
We've done our best to take the technical high road in those hard decisions - generally trading in ease of (our) development for stronger guarantees, more rigorous testing, and higher performance. We're excited to talk about our development process and decisions with the world. Looks like we've got some writing to do :)