The deployment pipeline is a small portion of it. Even capistrano can provide the same.

The problems you describe are definitely real and non-trivial, but Chef helps with this.

Heroku is also not a silver bullet for scaling up.