Subscribe to the ruby on rails security mailing list if you haven't already: https://groups.google.com/forum/?fromgroups#!forum/rubyonrai....

I also started using Gemnasium which does a good job of notifying me about security patches in my app's dependencies along with keeping me up to date with gem releases.