Heroku is definitely up to the job. I've stored 100s of thousands of tweets a day using Heroku workers and a larger Amazon RDS instance.
Have a look at scaling your Resque workers up for your daily task, then back down once you're done. I haven't used the following technique, but it looks sound:
Comments
Heroku is definitely up to the job. I've stored 100s of thousands of tweets a day using Heroku workers and a larger Amazon RDS instance.
Have a look at scaling your Resque workers up for your daily task, then back down once you're done. I haven't used the following technique, but it looks sound:
http://blog.darkhax.com/2010/07/30/auto-scale-your-resque-wo...
Actually already using this, works great!
Thanks for the input, I'll stick with Heroku for now!