Would strongly disagree with this. Not all parts of your entire stack are equal. It is far, far more likely that your application tier is likely to fall over or say suffer a full GC than your load balancer falling over. You've also eliminated the ability to do rolling restarts and many other redundancy activities.

Staying away from complexity is one thing. But only if you are running some toy site.