This is very interesting project, all the current DB are optimized for normal HD(and the standard HD is the slowest part of our PC). But with development of the Solid-State Drives we will have more and more fast drives. So the Database who will take advantage of the these new SSDrives will lead the way in Database design technology. It is right time to invest in this technology.
But really, rotating disks are not that bad for most database use cases. B-trees, the usual on-disk database structure, are designed to keep similar data on the same disk page, which means that if you request row 42, row 43 will be in memory by the time you need it. So the slowness of the disk is abstracted away; iterate over your data in index order, and it's always fast.
Hash tables have a theoretical advantage over balanced trees, and an SSD would make a naive hash table implementation easier to implement. But if you are smart (like, say, BerkeleyDB), hash tables and balanced trees have almost the same real world performance.
RethinkDB might be better for write-heavy operations, but that's because SSDs are better for random writes.
if you request row 42, row 43 will be in memory by the time you need it
True, but this is rarely the case for OLTP workloads. What happens when there is a credit card transaction with a user ID 100731, followed by a credit card transaction with a user ID 8762592? Even for range queries, what you're saying is true only if you're walking through the primary index. The second you start walking through the secondary indices, you're back to random read land (my Facebook friends, for example, are extremely unlikely to be stored in the user table sequentially).
SSDs are better for random writes
Random writes are very tricky on SSDs because of the slow erase operation. The FTL controllers are getting much better at this on micro benchmarks, but it's very difficult to measure random write performance profile over different timelines and different disk space utilization scenarios.
I'm not convinced. Modern OSes use locking extensively and do perfectly fine on multicore applications (FreeBSD's pgsql performance scales linearly up to 16 cores last time I saw graphs).
Obviously you need to be smart about how you do your locking (no giant lock!) but the mere fact of having locking is not automatically a problem.
I misspoke, I should have said 'many-core'.
Yes, you're probably right that no respectable database is going to have a problem with lock contention on 16 cores. But, AMD released 12 core processors this week. Its likely we'll see the average DB server have 48 cores sometime in the next year or two, and who knows after that.
I quoted 16 cores because that's the biggest hardware the FreeBSD project had available when those benchmarks were being run -- I suspect that it scales linearly quite a bit further than that.
I read over their page more carefully, and I think I see what they mean by "no locks". It means that the database stays internally consistent regardless of read or write order. When you start a transaction, you see the data in the log before you started, but you don't see any changes after you start. Fine.
You can get more isolation that this, and you need to to really keep your data consistent, but all DBs except Berkeley seem to have this off by default. So I am not too bothered by this, but I would be interested in seeing how well Rethink handles concurrent OLTP applications that actually care about data integrity. Caring about data integrity is slow, and Rethink might not speed this up all that much. Or it might :)
Comments
This is very interesting project, all the current DB are optimized for normal HD(and the standard HD is the slowest part of our PC). But with development of the Solid-State Drives we will have more and more fast drives. So the Database who will take advantage of the these new SSDrives will lead the way in Database design technology. It is right time to invest in this technology.
But really, rotating disks are not that bad for most database use cases. B-trees, the usual on-disk database structure, are designed to keep similar data on the same disk page, which means that if you request row 42, row 43 will be in memory by the time you need it. So the slowness of the disk is abstracted away; iterate over your data in index order, and it's always fast.
Hash tables have a theoretical advantage over balanced trees, and an SSD would make a naive hash table implementation easier to implement. But if you are smart (like, say, BerkeleyDB), hash tables and balanced trees have almost the same real world performance.
RethinkDB might be better for write-heavy operations, but that's because SSDs are better for random writes.
if you request row 42, row 43 will be in memory by the time you need it
True, but this is rarely the case for OLTP workloads. What happens when there is a credit card transaction with a user ID 100731, followed by a credit card transaction with a user ID 8762592? Even for range queries, what you're saying is true only if you're walking through the primary index. The second you start walking through the secondary indices, you're back to random read land (my Facebook friends, for example, are extremely unlikely to be stored in the user table sequentially).
SSDs are better for random writes
Random writes are very tricky on SSDs because of the slow erase operation. The FTL controllers are getting much better at this on micro benchmarks, but it's very difficult to measure random write performance profile over different timelines and different disk space utilization scenarios.
But the lack of locking is potentially big for multicore applications.
I'm not convinced. Modern OSes use locking extensively and do perfectly fine on multicore applications (FreeBSD's pgsql performance scales linearly up to 16 cores last time I saw graphs).
Obviously you need to be smart about how you do your locking (no giant lock!) but the mere fact of having locking is not automatically a problem.
I misspoke, I should have said 'many-core'. Yes, you're probably right that no respectable database is going to have a problem with lock contention on 16 cores. But, AMD released 12 core processors this week. Its likely we'll see the average DB server have 48 cores sometime in the next year or two, and who knows after that.
I quoted 16 cores because that's the biggest hardware the FreeBSD project had available when those benchmarks were being run -- I suspect that it scales linearly quite a bit further than that.
So that data corruption can scale linearly?
(Disks vs. SSDs and transactions vs. "eventual consistency" are orthogonal.)
Lock-free != data corruption. You just use atomic builtins[1] as primitives, instead of mutexes and semaphores.
[1] http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins....
Are we talking about low-level or high-level locking? I am talking about high-level locking semantics, like transactions.
At least as they've announced it so far, RethinkDB is a back-end for MySQL; so you should still have all of MySQL's transactional functionality.
(I'm sure Slava will correct me if I'm wrong here...)
I read over their page more carefully, and I think I see what they mean by "no locks". It means that the database stays internally consistent regardless of read or write order. When you start a transaction, you see the data in the log before you started, but you don't see any changes after you start. Fine.
You can get more isolation that this, and you need to to really keep your data consistent, but all DBs except Berkeley seem to have this off by default. So I am not too bothered by this, but I would be interested in seeing how well Rethink handles concurrent OLTP applications that actually care about data integrity. Caring about data integrity is slow, and Rethink might not speed this up all that much. Or it might :)
Oracle doesn't even support repeatable read (which seems weird to me). Read Commited should be doable in their system.
Oracle is marketing, not software :)