You have the full HN dataset and voting logs. What if you used it to train a simple classifier that would predict the likely karma of a comment as a function of a number of features, including username, username responded to, bag-of-words vector of the comment, time of day, total number of comments in thread, "controversy" of the comment (as reflected by a mix of up & down votes) and similar factors?
The goal could be something like Google's Beer Goggles, but for comments: "it looks like you're writing a flame, are you sure you want to do that?"
Another thought: there's a big difference between a comment at -1 with 10 upvotes and 12 downvotes vs. one that has occasioned 2 downvotes. Might be interesting to see the effect of showing both the number of up & down votes rather than just the sum. The idea is that genuinely trollish comments at -1 will be differentiated from minority points of view.
Indeed, the idea is that a "controversial" comment can evoke both disagreement and support, while a trollish comment would just result in downvoting.
OTOH, some particularly clever trolls are of course intended to provoke division of an intellectually unproductive sort. Not sure if machine learning could identify that...though features computed from the commenter's past history could be informative.
I think it's more effective to train classifiers on users' voting logs and comments in order to maintain the quality and thoughtfulness of HN.
This allows you to classify all your users into classes that resemble their characters.
e.g. A genuine nice guy won't post mean comments and won't pick 'punish' a user he disagrees with.
A person who downvotes when he disagrees can be classified as such.
I'm sure a troll has a specific pattern too.
Then you can enable or disable aspects of the site for each user-category.
e.g. disable comments from trolls, disable downvoting for people who downvote when disagree as this stimulates groupthink, etc...
A set of users can serve as a training set for each category.
It is possible to let the 'genuine'-category classify comments from users from the other categories in order to enhance the classifiers.
Actually this is an interesting machine learning problem, I still need to pick a project for my course in machine learning so I can research this if I can get my hands on the datasets.
Comments
You have the full HN dataset and voting logs. What if you used it to train a simple classifier that would predict the likely karma of a comment as a function of a number of features, including username, username responded to, bag-of-words vector of the comment, time of day, total number of comments in thread, "controversy" of the comment (as reflected by a mix of up & down votes) and similar factors?
The goal could be something like Google's Beer Goggles, but for comments: "it looks like you're writing a flame, are you sure you want to do that?"
Another thought: there's a big difference between a comment at -1 with 10 upvotes and 12 downvotes vs. one that has occasioned 2 downvotes. Might be interesting to see the effect of showing both the number of up & down votes rather than just the sum. The idea is that genuinely trollish comments at -1 will be differentiated from minority points of view.
I hope you would reward controversy rather than punish it? I don't come to HN to read happythoughts.
Indeed, the idea is that a "controversial" comment can evoke both disagreement and support, while a trollish comment would just result in downvoting.
OTOH, some particularly clever trolls are of course intended to provoke division of an intellectually unproductive sort. Not sure if machine learning could identify that...though features computed from the commenter's past history could be informative.
I think it's more effective to train classifiers on users' voting logs and comments in order to maintain the quality and thoughtfulness of HN. This allows you to classify all your users into classes that resemble their characters. e.g. A genuine nice guy won't post mean comments and won't pick 'punish' a user he disagrees with. A person who downvotes when he disagrees can be classified as such. I'm sure a troll has a specific pattern too.
Then you can enable or disable aspects of the site for each user-category. e.g. disable comments from trolls, disable downvoting for people who downvote when disagree as this stimulates groupthink, etc...
A set of users can serve as a training set for each category. It is possible to let the 'genuine'-category classify comments from users from the other categories in order to enhance the classifiers.
Actually this is an interesting machine learning problem, I still need to pick a project for my course in machine learning so I can research this if I can get my hands on the datasets.