On those simple ideas to fix it: I don't think it's simple. Once you do simple heuristics, the other side will start doing it just a little bit more sophisticated, to get around the simple heuristics. So, then you improve the heuristics to catch the spammer. And again, the spammers get around that improved tests as well. And so on.
In the end, you end up with similar spam filter methods as we also have for mails and probably as other social networks have as well. But this is far from simple. I don't think having a huge number of hand-crafted heuristics is really a good solution. I think it should be machine learning model which you train and it does it all automatically without too much false positives (and also not too much false negatives).
I think issues like these will continue along the path you describe until we tie authorship to real humans and their reputation, or perhaps something else they care about, in one way or another.
I'm not saying that we should give up on the idea of anonymity on the internet, at least not completely. But real humans have to put something at stake when they use space on any part of the internet that other humans should care about.
I've participated in plenty of communities without spam problems where I have no idea of the real identity of the other participants, and neither does anyone else.
I can't help but note that neither of us attach our real identity to our participation here.
I think a better solution might be structural. In this GitHub example, why can someone tag people who they're not collaborating with to begin with? Why are identities so easily discoverable and contactable? The value seems to far outweigh the risk.
I hope there are other ways. Although I suspect that the communities you're thinking about are somewhat niche? I guess that would only motivate a small amount of investment for setting up spambots and a little resistance is enough to move on. If this explanation is not enough I also wonder what the explanation would be?
I also suspect that the barrier for spamming absolutely everywhere is going to get lowered as soon as somebody figures out how to make LLM:s configure spambots for arbitrary domains?
I don't understand your comment; if reddit is part of the dead internet then it is full of spam bots no? Or are you saying that users perceive that most other users are bot when they are not?
I think issues like these will continue along the path you describe until we tie authorship to real humans and their reputation, or perhaps something else they care about, in one way or another.
I think this is attacking the problem from the wrong end. This is just increasing the penalty for spamming. (Real reputational damage.)
Why not shift the other end—increase the difficulty of spamming in the first place.
Could be as simple as charging for account creation (even if just a deposit that’s refunded after some amount of non-spam activity).
But without getting into all the baggage payments bring—don’t allow people to ping others until they’ve reached a certain “reputation” by hitting some threshold of non-spam activity? Rate limit how many people you can ping based on reputation? Limit to only users associated with the repo you’re making the MR on? Rate limit how quickly you can create MRs and comments based on reputation?
Seems like there are a lot of levers to pull here that could curtail the problem without jumping straight to tying accounts to real life IDs.
A one time administration fee for account creations would make most of these spam tactics unprofitable. People are probably too used to having everything for free online (paid for by ads) for anyone to try this right now.
The fix isn't ever more heuristics - that is an uphill battle that can't ever be won. The fix is following the money and disconnecting the bad actors' ISPs from the Internet.
There was a time when abuse@<isp-domain.tld> emails were honored and administrators actually took notice of what came in, but these days are long since gone - ISPs simply don't want to spend the money, and so the cost of abuse is externalized to society at large.
ETA: Also, a fix would be to have a human with more than ten seconds time take a look at even 1% of spam reports. Spammers are lazy, they always use the same template, so if you have a human actually looking into the template and then routing every match to /dev/null, it's far more effective. Like... I can do this on Twitter for every new variation of some scam, why can't Twitter do it on its own?!
Do you really want ISPs to be in the business of deciding what should and should not be on the internet though? That sort of thing typically doesn't work too well.
Deciding what is or isn't a "scam" is really a job for the independent judiciary. But getting a ruling is difficult and time-consuming, and also largely pointless because a new website can be created almost instantly, and there are many foreign ISPs where you can't get a ruling at all.
I don't really disagree with your basic premise that "disconnecting the bad actors' ISPs from the Internet" is the ideal solution, but this is far more difficult than your comment implies – almost impossible with how the internet currently works.
I think that the parent comment was referring to the days when ISPs were local or network services were provided by a school or a some other entity which had a vested interest in keeping their network clear of bad actors.
Author/OP here: ya "simple" is a relative word and I tried to address that with a few sentences around it. Didn't want to come across as an "internet arm chair expert" since I don't have a lot of experience in at scale content moderation.
On the other hand, I don't think posting the same spam nearly 1k times in 24 hours shouldn't raise some sort of alarm. 1 manual take down of a spam by a human should trigger a search (or have an option) to trigger a search through recent comments to see ones that are close. If I can search their whole site in 3 seconds they should have some sort of system that looks for 95% simular comments.
Seems simple but I am sure it's actually much much harder and has caveats and gotchas along the way as it scales.
I mean, you suggest using AI to scan comments/code.
Maybe think harder and don't avoid a certain tech that is in a bubble and may soon be sued out of existence.
I do agree, github and other places that allow user generated content need to do a better job. I have more bots than humans following on twitter (and I have thousands of followers), for example, however trying to say "AI CAN FIX IT" won't win you any favors.
FYI I have a suggestion, maybe spin up said AI model on AWS and try scanning all the comments/issues github receives in a day. When even Microsoft can't pay your bill, you'll see why they do not yet use AI for that...though I'm sure they will try eventually.
Yes it's a game of cat and mouse, but starting out with a defeatist attitude, the cat makes things way too easy and the mice will simply roam free. Making life hard for the spammers can and will get rid of all but the most persistent.
The other issue I have with this particular GitHub spam is the notifications persist even after the spam has been removed. You get notified and subscribed to some random thread because you were previously tagged in it.
After GH removes the spam (which is currently too slow) they should also retract any notifications or subscriptions that were made as a result of the spam comment.
That's right. If you login to to GitHub even days after the spam is removed, you will see the bell icon with a notification about a since cleaned up thread in your feed.
Comments
On those simple ideas to fix it: I don't think it's simple. Once you do simple heuristics, the other side will start doing it just a little bit more sophisticated, to get around the simple heuristics. So, then you improve the heuristics to catch the spammer. And again, the spammers get around that improved tests as well. And so on.
In the end, you end up with similar spam filter methods as we also have for mails and probably as other social networks have as well. But this is far from simple. I don't think having a huge number of hand-crafted heuristics is really a good solution. I think it should be machine learning model which you train and it does it all automatically without too much false positives (and also not too much false negatives).
I think issues like these will continue along the path you describe until we tie authorship to real humans and their reputation, or perhaps something else they care about, in one way or another.
I'm not saying that we should give up on the idea of anonymity on the internet, at least not completely. But real humans have to put something at stake when they use space on any part of the internet that other humans should care about.
I've participated in plenty of communities without spam problems where I have no idea of the real identity of the other participants, and neither does anyone else.
I can't help but note that neither of us attach our real identity to our participation here.
I think a better solution might be structural. In this GitHub example, why can someone tag people who they're not collaborating with to begin with? Why are identities so easily discoverable and contactable? The value seems to far outweigh the risk.
I hope there are other ways. Although I suspect that the communities you're thinking about are somewhat niche? I guess that would only motivate a small amount of investment for setting up spambots and a little resistance is enough to move on. If this explanation is not enough I also wonder what the explanation would be?
I also suspect that the barrier for spamming absolutely everywhere is going to get lowered as soon as somebody figures out how to make LLM:s configure spambots for arbitrary domains?
Reddit, for instance, in my experience, has been remarkably spam bot free, though the dead internet theory is very prevalent in culture there
I don't understand your comment; if reddit is part of the dead internet then it is full of spam bots no? Or are you saying that users perceive that most other users are bot when they are not?
I think this is attacking the problem from the wrong end. This is just increasing the penalty for spamming. (Real reputational damage.)
Why not shift the other end—increase the difficulty of spamming in the first place.
Could be as simple as charging for account creation (even if just a deposit that’s refunded after some amount of non-spam activity).
But without getting into all the baggage payments bring—don’t allow people to ping others until they’ve reached a certain “reputation” by hitting some threshold of non-spam activity? Rate limit how many people you can ping based on reputation? Limit to only users associated with the repo you’re making the MR on? Rate limit how quickly you can create MRs and comments based on reputation?
Seems like there are a lot of levers to pull here that could curtail the problem without jumping straight to tying accounts to real life IDs.
A one time administration fee for account creations would make most of these spam tactics unprofitable. People are probably too used to having everything for free online (paid for by ads) for anyone to try this right now.
The fix isn't ever more heuristics - that is an uphill battle that can't ever be won. The fix is following the money and disconnecting the bad actors' ISPs from the Internet.
There was a time when abuse@<isp-domain.tld> emails were honored and administrators actually took notice of what came in, but these days are long since gone - ISPs simply don't want to spend the money, and so the cost of abuse is externalized to society at large.
ETA: Also, a fix would be to have a human with more than ten seconds time take a look at even 1% of spam reports. Spammers are lazy, they always use the same template, so if you have a human actually looking into the template and then routing every match to /dev/null, it's far more effective. Like... I can do this on Twitter for every new variation of some scam, why can't Twitter do it on its own?!
Do you really want ISPs to be in the business of deciding what should and should not be on the internet though? That sort of thing typically doesn't work too well.
Deciding what is or isn't a "scam" is really a job for the independent judiciary. But getting a ruling is difficult and time-consuming, and also largely pointless because a new website can be created almost instantly, and there are many foreign ISPs where you can't get a ruling at all.
I don't really disagree with your basic premise that "disconnecting the bad actors' ISPs from the Internet" is the ideal solution, but this is far more difficult than your comment implies – almost impossible with how the internet currently works.
We are oh so close to a balkanised internet.
Thankfully, the powers that be outside of CN, RU and a few others haven't really gone to town on buggering up basic connectivity, yet.
I think that the parent comment was referring to the days when ISPs were local or network services were provided by a school or a some other entity which had a vested interest in keeping their network clear of bad actors.
The fix is to give users the tools to lean more heavily on transitive trust and less heavily on the platform.
Or we build those tools ourselves and let the platforms be dumb pipes which we selectively slurp from.
The days of trusted-by-default banned-selectively are ending. It's dark forest time.
This ignores that botnets, malware, etc exist.
Well, ISPs can hold their customers accountable as well. Get told you're running malware, you got 24h to get it fixed, or you get disconnected.
Author/OP here: ya "simple" is a relative word and I tried to address that with a few sentences around it. Didn't want to come across as an "internet arm chair expert" since I don't have a lot of experience in at scale content moderation.
On the other hand, I don't think posting the same spam nearly 1k times in 24 hours shouldn't raise some sort of alarm. 1 manual take down of a spam by a human should trigger a search (or have an option) to trigger a search through recent comments to see ones that are close. If I can search their whole site in 3 seconds they should have some sort of system that looks for 95% simular comments.
Seems simple but I am sure it's actually much much harder and has caveats and gotchas along the way as it scales.
I mean, you suggest using AI to scan comments/code.
Maybe think harder and don't avoid a certain tech that is in a bubble and may soon be sued out of existence.
I do agree, github and other places that allow user generated content need to do a better job. I have more bots than humans following on twitter (and I have thousands of followers), for example, however trying to say "AI CAN FIX IT" won't win you any favors.
FYI I have a suggestion, maybe spin up said AI model on AWS and try scanning all the comments/issues github receives in a day. When even Microsoft can't pay your bill, you'll see why they do not yet use AI for that...though I'm sure they will try eventually.
Yes it's a game of cat and mouse, but starting out with a defeatist attitude, the cat makes things way too easy and the mice will simply roam free. Making life hard for the spammers can and will get rid of all but the most persistent.
The other issue I have with this particular GitHub spam is the notifications persist even after the spam has been removed. You get notified and subscribed to some random thread because you were previously tagged in it.
After GH removes the spam (which is currently too slow) they should also retract any notifications or subscriptions that were made as a result of the spam comment.
How does that work with email?
I think he means the notification model .
You can disable email notifications, you cannot disable the notifications model .
That's right. If you login to to GitHub even days after the spam is removed, you will see the bell icon with a notification about a since cleaned up thread in your feed.