For about a year or two, I've had set -o vi in my .bashrc. It is just so much easier when you work in Vim more than half the time!

The modes still trip me up sometimes, but a quick <Esc> or <Ctrl+c> fixes that!