I'm afraid I don't understand that second part of your comment - it's not clear to me what distinction you're wanting to appear in the UI. But I can answer this part:
if more people downvote / individual flag it than upvote it, then how did it rise to the front page to start with?
It's because most of the flags only happen later. Upvotes get a post onto the frontpage, then other users see it on the frontpage and think "wtf? that doesn't belong on HN" and flag it. This is especially common with sensational or outrageous stories, since those qualities inevitably attract upvotes—but then later they attract flags. You can think of flags as a bit like an immune system in that sense.
Comments
I'm afraid I don't understand that second part of your comment - it's not clear to me what distinction you're wanting to appear in the UI. But I can answer this part:
It's because most of the flags only happen later. Upvotes get a post onto the frontpage, then other users see it on the frontpage and think "wtf? that doesn't belong on HN" and flag it. This is especially common with sensational or outrageous stories, since those qualities inevitably attract upvotes—but then later they attract flags. You can think of flags as a bit like an immune system in that sense.