Filtering duplicate URIs isn't enough to get rid of duplicate submissions. There are often several forms of a URI for the same page, and the same writeup is published on several news sites, never mind similar takes on the issue at hand by multiple bloggers.
This isn't something an algorithm can solve, but a good UI for searching old stories and annotating submissions ("this is the eyewitness take on that") could be worth it to reduce the amount of perceived redundancy.
Comments
Filtering duplicate URIs isn't enough to get rid of duplicate submissions. There are often several forms of a URI for the same page, and the same writeup is published on several news sites, never mind similar takes on the issue at hand by multiple bloggers.
This isn't something an algorithm can solve, but a good UI for searching old stories and annotating submissions ("this is the eyewitness take on that") could be worth it to reduce the amount of perceived redundancy.