You're definitely not crazy. I am not as far into my own career as you, but I would even say that it seems the crazy thing to do, in your position, is to do nothing. You want more from life? Get it! You will be able to find a job later, guaranteed, especially if you kept hacking after leaving your big-co job! (Do re-adjust expectations, though. A start-up will not pay you nearly as much as your last job did, although you should get stock. You know all this, but it will still be a big change.)