Adafruit, Sparkfun, Pololu, Seeed Studios, Leaf Labs Also see Mouser, Digikey, Allied, et al http://leaflabs.com/distributors/ Radio Shack carries Arduino now, some do anyway. My personal favorite is Adafruit because this one time, I whined to her about her webform not accepting my name+spamtagger@emailaddress.com and she fixed it and wrote me back in like five minutes, no joke. I do not know her personally and that was the first time I had ordered anything from her.

tl;dr Limor Fried is a bad ass, buy things from her store.

You can get stuff dirt cheap straight from Hong Kong on eBay with free shipping.

I pay about half the price of what I see in most web stores.

If you are new to electronics I would suggest the arduino inventor kit. Comes with everything you need to get your feet wet.

Sparkfun.