The only drawback i personally see is in fact that, when you realize that "everything is your fault" ie, that you can influence things, people around you to obtain the desired result, is in fact manipulation.

And I don't like manipulating people. Yes, it's a lot easier. yes, leaders do that non-stop. But personally I can't get around to doing that.

Also, if everyone was doing it, we'd probably play a much higher level of "manipulation game" and feel fake about everything we do.