I gave up on reading every single tweet, just as I gave up on reading every RSS entry in my reader. I just check it when I get bored or need a break for a minute or two.

I probably don't spend more than 15 minutes a day dealing with Twitter, but I still like using it.