1) Take Redis simple enough so that a single developer can understand the implementation in little time.
2) Run Redis on my own large sites. The problem is that while this sites are really large they are famous only in Italy. One example is http://oknotizie.alice.it
3) In the future when Redis will be stable and some little startup will start using it for real work try to collect success stories and write it in the front page of the project.
I've been evaluating databases recently. It doesn't matter if the site is not well-known in the US as long as you can post traffic numbers. TokyoTyrant is Japanese, after all.
Also these help:
* able to handle large (>200GB) datasets
* client libraries for top N languages
* easy way to write client libs (eg simple protocol, a C library, etc)
* connection pooling and/or cheap connections
* easy to install on Mac, Linux and Windows development machines
* repl (this is not much of a problem with Python & Ruby)
* stable dump / restore format
One problem for you is that MySQL plus serialize() basically does these already, with 10+ years of testing on top. Your system has to do a lot more to make it worth the risk.
Scale enough in which way? Read traffic? Write traffic? Locking? If you use MySQL as a btree it is pretty fast and consistent. I gather from this post that a lot of companies started in the last 18 months are doing this. It's a hack but a remarkably useful one. Good luck! I will keep my eye on redis.
Probably it just depends on the dataset. The issue we have with our service (http://lloogg.com) is that you need to take the last N items of logs for every site. To ask for the latest M items should be fast. To push a new log line on the list should be fast. Every kind of MySQL configuration we tried was unable to reach the 10k writes/second we reach with Redis. Obviously. Even when you use MySQL as a btree implementation you get a lot of overhead. Starting from the protocol and the format of the statements, for example.
The idea to encode things with json or other formats in a blog text is just a ugly hack. People are using this because they are desperate, not because is good computer science. They started with mysql, know mysql, hacked with mysql. Clearly will try to fix their site with MySQL.
The json+blob can work as long as the data that's stored in this fields is trivial to serialize-deserialize. What about having a 10000 elements list in every blob and at every page view you need to append an element?
So: great hack, you found a way to work with the tools you have, but this does not mean in any way that fast key-value persistent DBs don't have something to say into the web-scale theater.
Comments
ok this sounds like I can have this goals:
1) Take Redis simple enough so that a single developer can understand the implementation in little time.
2) Run Redis on my own large sites. The problem is that while this sites are really large they are famous only in Italy. One example is http://oknotizie.alice.it
3) In the future when Redis will be stable and some little startup will start using it for real work try to collect success stories and write it in the front page of the project.
I've been evaluating databases recently. It doesn't matter if the site is not well-known in the US as long as you can post traffic numbers. TokyoTyrant is Japanese, after all.
Also these help:
One problem for you is that MySQL plus serialize() basically does these already, with 10+ years of testing on top. Your system has to do a lot more to make it worth the risk.Agreed on most of the points... about MySQL, I developed Redis just because MySQL does not scale enough with given kind of datasets :)
Scale enough in which way? Read traffic? Write traffic? Locking? If you use MySQL as a btree it is pretty fast and consistent. I gather from this post that a lot of companies started in the last 18 months are doing this. It's a hack but a remarkably useful one. Good luck! I will keep my eye on redis.
Probably it just depends on the dataset. The issue we have with our service (http://lloogg.com) is that you need to take the last N items of logs for every site. To ask for the latest M items should be fast. To push a new log line on the list should be fast. Every kind of MySQL configuration we tried was unable to reach the 10k writes/second we reach with Redis. Obviously. Even when you use MySQL as a btree implementation you get a lot of overhead. Starting from the protocol and the format of the statements, for example.
The idea to encode things with json or other formats in a blog text is just a ugly hack. People are using this because they are desperate, not because is good computer science. They started with mysql, know mysql, hacked with mysql. Clearly will try to fix their site with MySQL.
The json+blob can work as long as the data that's stored in this fields is trivial to serialize-deserialize. What about having a 10000 elements list in every blob and at every page view you need to append an element?
So: great hack, you found a way to work with the tools you have, but this does not mean in any way that fast key-value persistent DBs don't have something to say into the web-scale theater.