I like this article: while it is for sure not the definitive guide to NoSQL, it is a short description mostly about facts that people new to the field can use to get an idea about what a good candidate could be for initial experimentation, given a defined problem to solve.
That said I think that picking the good database is something you can do only with a lot of work. Picking good technologies for your project is hard work, so there is to try one, and another and so forth, and even reconsidering after a few years (or months?) the state of the things again, given the evolution speed of the DB panorama in the recent years.
While I'm at it I like to share that in this exact days I'm working at a Redis disk back end. I've already a prototype working after a few days of full immersion (I like to use vacation time to work at completely new ideas for Redis).
The idea is that everything is stored on disk, in what is a plain key-value database (complex values are serialized when on disk), and the memory is instead used as an object cache.
It is like taking current Redis Virtual Memory and inverting the logic completely, the result is the same (working set in memory, the rest on disk), but this implementation means that there are no limits on the data you can put into a single instance, that you don't have slow restarts (data is not loaded on memory if not demanded), and there isn't to fork() to save. Keys marked as "dirty" (modified) are transfered to disk asynchronously as needed, by IO threads.
If everything will work as I expect (and initial tests are really encouraging) this means that Redis 2.4 will exit in a few months completely killing the current Virtual Memory implementation in favor of the new "two back ends" design, where you can select if you want to run an in-memory DB or an on-disk DB where memory is just an LRU cache for the working set.
antirez, it's a honor that you commented, thanks! :)
The new inverted logic for the VM you describe seems very interesting; I'm very much looking forward to see 2.4!
Redis is already more than perfect what we use it for -- keeping track of stock price data, and distributing it. The size of the DB is known in advance (the amount of stocks does not grow very fast), and the performance is perfect.
I think the main business of Redis is still as an in-memory DB / cache / messaging system and so forth. We have a decent implementation from this point of view, so the next logical step is making it working in a cluster.
On the other side it's really interesting to see what people can do with Redis data model if much larger datasets can be used without problems (at the cost of performances of course... can't be as fast as memory). VM was my first idea, but I need to admit, I don't like the design at this point. This new design can be much better, and we can have it production ready in a few months. So I'm curious about what will happen in 2011! :)
Comments
I like this article: while it is for sure not the definitive guide to NoSQL, it is a short description mostly about facts that people new to the field can use to get an idea about what a good candidate could be for initial experimentation, given a defined problem to solve.
That said I think that picking the good database is something you can do only with a lot of work. Picking good technologies for your project is hard work, so there is to try one, and another and so forth, and even reconsidering after a few years (or months?) the state of the things again, given the evolution speed of the DB panorama in the recent years.
While I'm at it I like to share that in this exact days I'm working at a Redis disk back end. I've already a prototype working after a few days of full immersion (I like to use vacation time to work at completely new ideas for Redis).
The idea is that everything is stored on disk, in what is a plain key-value database (complex values are serialized when on disk), and the memory is instead used as an object cache. It is like taking current Redis Virtual Memory and inverting the logic completely, the result is the same (working set in memory, the rest on disk), but this implementation means that there are no limits on the data you can put into a single instance, that you don't have slow restarts (data is not loaded on memory if not demanded), and there isn't to fork() to save. Keys marked as "dirty" (modified) are transfered to disk asynchronously as needed, by IO threads.
If everything will work as I expect (and initial tests are really encouraging) this means that Redis 2.4 will exit in a few months completely killing the current Virtual Memory implementation in favor of the new "two back ends" design, where you can select if you want to run an in-memory DB or an on-disk DB where memory is just an LRU cache for the working set.
antirez, it's a honor that you commented, thanks! :)
The new inverted logic for the VM you describe seems very interesting; I'm very much looking forward to see 2.4!
Redis is already more than perfect what we use it for -- keeping track of stock price data, and distributing it. The size of the DB is known in advance (the amount of stocks does not grow very fast), and the performance is perfect.
Keep up the good job! (And have a nice new year)
Kristof
Thank you kkovacs!
I think the main business of Redis is still as an in-memory DB / cache / messaging system and so forth. We have a decent implementation from this point of view, so the next logical step is making it working in a cluster.
On the other side it's really interesting to see what people can do with Redis data model if much larger datasets can be used without problems (at the cost of performances of course... can't be as fast as memory). VM was my first idea, but I need to admit, I don't like the design at this point. This new design can be much better, and we can have it production ready in a few months. So I'm curious about what will happen in 2011! :)
Thank you and have a nice new year as well.
Thanks :)
Picking good technologies for your project is hard work
This is probably the most important and relevant point I've seen in a while. Architects should take note...
This sounds like the approach taken by Membase (www.membase.org). And it is an incredibly powerful system. Not a bad model to follow.