Actually, we do run redis in production, and it has been awesome for us. But, for this particular aspect of our application, we see about 20GB of data growth per week. So, an all-in-RAM db isn't really practical.
Either way, durability wasn't the issue for us. Mongodb has no write concurrency. So, when it got so overloaded with data that queries were taking forever, we had no way to purge older data without blocking all read & write operations for hours.
Ok sorry for assuming that the problem was durability / reliability.
If MongoDB has no write concurrency MySQL is simply better from this point of view for your use case as far as I understand, but I'm not very good at MongoDB.
Btw Redis will get Virtual Memory in a few months, I hope you can reconsider it later ;)
Comments
Actually, we do run redis in production, and it has been awesome for us. But, for this particular aspect of our application, we see about 20GB of data growth per week. So, an all-in-RAM db isn't really practical.
Either way, durability wasn't the issue for us. Mongodb has no write concurrency. So, when it got so overloaded with data that queries were taking forever, we had no way to purge older data without blocking all read & write operations for hours.
Ok sorry for assuming that the problem was durability / reliability.
If MongoDB has no write concurrency MySQL is simply better from this point of view for your use case as far as I understand, but I'm not very good at MongoDB.
Btw Redis will get Virtual Memory in a few months, I hope you can reconsider it later ;)
This is fixed now - large deletes won't block the world forever: http://jira.mongodb.org/browse/SERVER-494
Awesome. You guys rock!
Or if you want to only keep x amount of data around - thats what capped collections were designed for.
Did you try having multiple collections and then just dropping them?