I think it's a function of there being too many items submitted every day, and not enough people checking the "new" page. During busy times of the day, if you don't get 5-10 votes within the first 10-15 mins, you're on the 2nd page of the "new" posts area, and you're never going to make it. I'm afraid the end result is that the only posts that will end up on the front-page are:
1. Where the author (domain) is well-known
2. Where the title is link-bait
3. Where the submitter tries to game the system
All of these are bad for the future of HN. I really like the way reddit handles this, of putting a few new items at the top of the homepage and rotating them, so that more people see them and have the chance to upvote. The way things are going, HN is destined to have a front-page full of only linkbait, spam, and/or posts by rockstars.
Another obvious cause of submissions falling too quickly off the "new" posts page is that there is only room for the latest 30 posts. Seems like we could fit another 3-5 posts on the "new" page if we removed the posts that have already hit the front page. Of course, this is only a minor band-aid that doesn't solve the root problems, as you pointed out.
I am really curious as to the answer to this question:
Of the stories that got more than 10 votes, how many of those (first) 10 votes came from upvotes as opposed to multiple submission.
My theory is that it frequently takes multiple submissions to get stuff in the front page so that it can be upvoted, rather than through upvotes from New. In which case the best story in the world might not do well if only one person finds it.
Another approach is to prevent these - which is how pg seems to describe his role at HN these days, at least for (3) gaming the system - leaving the field clear for the good stories. The flag link often takes care of (2) link-bait (though most people complain rather than click it). Well-known (3) author/domain probably isn't too bad, if the brand is justified, and even seems to be countered at times, eg. Seth Godin doesn't come up much anymore.
The big problem I see is industry gossip, within which I include: the crunchpad drama; iPhone/iPad dramas; Zuckerman's integrity; Bill Gates' integrity; Steve Jobs' integrity; Larry Ellison's integrity; today's psychoanalysis of one of our own members; even the Ring of Dark Angels/AngelGate.
Now, industry gossip is relevant to people in the industry. But it's not Hacker News (at least, not according to the guidelines.)
If there is a clear perception of what HN is, and enough people resolve to flag stories that don't fit that perception, then this problem goes away. We deserve the HN we get.
The other way is for multiple people to post the same link, which adds to the total. Maybe if people started submitting more, or submissions from high karma types had more initial karma, it would have a cumulative effect.
Another idea may be to increase the amount of time that a new post is "boosted" into the main page list. It seems like new posts fall to the 2nd page really quick. So it might be fixable by tweaking parameters.
There is a different solution to the reddit one, one that discourages block voting.
Perhaps there should be voting groups, essentially a cap on the population of voters.
Each time you hit a threshold, you randomly split the voters in two and then the articles you see only get the votes of the people in your group. So one site has multiple groups on the go at one time, all to stop the overwhelming nature of popularity drowning out the minority.
It stops block voting as all you can do is influence the group you are in rather than the community as a whole. There's a good chance on a big site you're not in the same group.
On the otherhand you could end up with groups that have wildly different front pages that don't suit your tastes. Perhaps to solve that there could even be something intelligent about it in that it switches you into groups with voting tendencies like your own, improving your perceived quality of the site.
Comments are a little troublesome, do you include them in the group voting or not? In one group a comment could be negative, while popular in another.
A random thought that occurred to me, I generally hate these regular meta threads of 'HN has got bad'.
Do you know what I saw today on HN? DHH and Joel Spolsky duking it out in the comments. Two of my favourite internet personalities arguing.
Wow. I tried to build a voting-based news-to-your-personal-taste site, but never did I think of this "groups" idea. I'm intrigued.
Idea: do it to HN as an A/B experiment: see how the content of the sites diverges. Actually the "/classic" url does this, sort of, by ranking articles not by total votes but by votes from users older than one year.
Yea it suggests really interesting experimental hypotheses: would the content in the different groups diverge or would it stay more or less the same? And would the degree of divergence change over time?
Comments
I think it's a function of there being too many items submitted every day, and not enough people checking the "new" page. During busy times of the day, if you don't get 5-10 votes within the first 10-15 mins, you're on the 2nd page of the "new" posts area, and you're never going to make it. I'm afraid the end result is that the only posts that will end up on the front-page are:
All of these are bad for the future of HN. I really like the way reddit handles this, of putting a few new items at the top of the homepage and rotating them, so that more people see them and have the chance to upvote. The way things are going, HN is destined to have a front-page full of only linkbait, spam, and/or posts by rockstars.Another obvious cause of submissions falling too quickly off the "new" posts page is that there is only room for the latest 30 posts. Seems like we could fit another 3-5 posts on the "new" page if we removed the posts that have already hit the front page. Of course, this is only a minor band-aid that doesn't solve the root problems, as you pointed out.
Getting dead ones off the page would yield another few.
You can set "showdead" to false in your profile.
I am really curious as to the answer to this question:
Of the stories that got more than 10 votes, how many of those (first) 10 votes came from upvotes as opposed to multiple submission.
My theory is that it frequently takes multiple submissions to get stuff in the front page so that it can be upvoted, rather than through upvotes from New. In which case the best story in the world might not do well if only one person finds it.
Another approach is to prevent these - which is how pg seems to describe his role at HN these days, at least for (3) gaming the system - leaving the field clear for the good stories. The flag link often takes care of (2) link-bait (though most people complain rather than click it). Well-known (3) author/domain probably isn't too bad, if the brand is justified, and even seems to be countered at times, eg. Seth Godin doesn't come up much anymore.
The big problem I see is industry gossip, within which I include: the crunchpad drama; iPhone/iPad dramas; Zuckerman's integrity; Bill Gates' integrity; Steve Jobs' integrity; Larry Ellison's integrity; today's psychoanalysis of one of our own members; even the Ring of Dark Angels/AngelGate. Now, industry gossip is relevant to people in the industry. But it's not Hacker News (at least, not according to the guidelines.)
If there is a clear perception of what HN is, and enough people resolve to flag stories that don't fit that perception, then this problem goes away. We deserve the HN we get.
In some cases, the timing of the submission also matters.
e.g I've posted a few interesting articles that dropped off the new page
The other way is for multiple people to post the same link, which adds to the total. Maybe if people started submitting more, or submissions from high karma types had more initial karma, it would have a cumulative effect.
Another idea may be to increase the amount of time that a new post is "boosted" into the main page list. It seems like new posts fall to the 2nd page really quick. So it might be fixable by tweaking parameters.
There is a different solution to the reddit one, one that discourages block voting.
Perhaps there should be voting groups, essentially a cap on the population of voters.
Each time you hit a threshold, you randomly split the voters in two and then the articles you see only get the votes of the people in your group. So one site has multiple groups on the go at one time, all to stop the overwhelming nature of popularity drowning out the minority.
It stops block voting as all you can do is influence the group you are in rather than the community as a whole. There's a good chance on a big site you're not in the same group.
On the otherhand you could end up with groups that have wildly different front pages that don't suit your tastes. Perhaps to solve that there could even be something intelligent about it in that it switches you into groups with voting tendencies like your own, improving your perceived quality of the site.
Comments are a little troublesome, do you include them in the group voting or not? In one group a comment could be negative, while popular in another.
A random thought that occurred to me, I generally hate these regular meta threads of 'HN has got bad'.
Do you know what I saw today on HN? DHH and Joel Spolsky duking it out in the comments. Two of my favourite internet personalities arguing.
That rocked. HN is still the best place to go.
Wow. I tried to build a voting-based news-to-your-personal-taste site, but never did I think of this "groups" idea. I'm intrigued.
Idea: do it to HN as an A/B experiment: see how the content of the sites diverges. Actually the "/classic" url does this, sort of, by ranking articles not by total votes but by votes from users older than one year.
Yea it suggests really interesting experimental hypotheses: would the content in the different groups diverge or would it stay more or less the same? And would the degree of divergence change over time?
I wonder how strong the correlation would be.
Now I think about it in the light of day it doesn't solve the problem of too many articles getting submitted.