This would solve the problem when a submission will get 20 or 30 upvotes just because of an interesting title, then most of the next 100 people looking at it would not agree but it's already got enough votes to sit on the front page for quiet a few hours.
Comments
Simplest solution: add down-voting to submissions as well as comments. Then people can downvote the super off-topic and repeats.
This would solve the problem when a submission will get 20 or 30 upvotes just because of an interesting title, then most of the next 100 people looking at it would not agree but it's already got enough votes to sit on the front page for quiet a few hours.
It does not solve the community-drift problem.
Unless you keep raising karma thresholds for moderation and posting to keep the where-is-the-next-Digg crowd out.