Skip to content

Comment on Ask PG: Would it be possible to do an A/B test on HN?parent

Comments

How many comments there are could be visible without any detrimental effect.

I simply suspect that the 'points' system has a shadow side and that as the site grows the shadow side starts to overpower the positive portion. By temporarily switching it off and asking the community what they felt about being 'point blind' for a short period and if they thought the quality improved or not you can make a 'metric' that is much easier to measure than some of the more technical tricks you could pull:

  Customer Satisfaction.
A simple poll after the experiment would suffice. Or you could make it switcheable on a user basis if it is a toss-up or too close to call a very clear preference.

If on average more people feel better without the points visible than with them after a short trial period then it's something that you could consider doing permanently.

Another option would be to keep author and points hidden until after you've voted for a comment, but that may have other side effects.

I think the content should stand on its own, regardless of what the voting history and the author are it is what you think about it that counts.

Right now the choice to vote or not to vote is made plenty of time based on the current number of votes, which leads to plenty of feedback loops. I've seen 'flip-flops' (bi stable and tri-stable, 0,1 and -1,0,1), positive feedback loops, and negative feedback loops.

By breaking the loop we could end up with a more balanced view.

Imagine what the effect would be of a running tally during an election, it would completely affect the outcome, and not necessarily in a positive way.

Again, it's just a gut feeling but I think there is some truth to it, and it's a very easy to do experiment, worst case we will learn that it did not work.

Thank you for further elaborating your idea.

I think the experiment would be a great idea and doing it the way you propose with empirical data to back up the machine generated statistics could provide very valuable insights.

Also one could start experimenting with the approach/idea e.g. let the number of votes appear after voting or not displaying a number at all

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.