I had an idea, similar to your suggestion, to vote on the link and the discussion separately. Although my idea would not separate children from adults (an idea I like very much), it would allow good links to rise without endorsing snarky and low-quality commentary on the link. Likewise, readers would feel more free to upvote quality discussion without necessarily endorsing the subject.

It is too bad that more than a couple voting toggles would be too cumbersome; otherwise both ideas could co-exist as four sets of up and down arrows.