People that have worked in start-ups are very well suited to being trouble-shooters in a large organization, and people that have worked for a big company stand a chance of running a start-up that interfaces / sells to big companies because they can relate to the party on the other side.

I'd think that it's a net win to have experience about the other side of the table, and that your skills are actually worth more by having been 'on the other side of the line' for a while.