i tend to agree, and i follow the mantra of only dishing out karma for things that are either particularly helpful/insightful/interesting, and only downvoting things that are actually detracting from the topic at hand.

everything else is neutral, which would, to me, include something that i don't agree with but doesn't actually detract from anything.