Choosing the path of least resistance

More often than not I've found myself regretting that I did not choose the path of least resistance; Spending years at a university to eventually get a job as a programmer when I could still have pursued my passion for programming in my free time with a more menial day job that did not require me to take home all that mental luggage and stress that comes with deadlines, when your profession is to be a programmer. In my country the pay would not be substantially less either and I could for example take a job doing manual labor for the added benefit of staying in shape(now I have to work out in addition to work).

I'm sure some of you feel the same.