So Facebook is presented with a trolley problem that have people lives on one track and more profit on the other track, and chose the latter, just as what we can expect from big corporations nowadays.
Not an unexpected decision overall, but it did surprise me how low Facebook can stoop. Like, they didn't even bother to fix their stuff in Myanmar after all these years with Myanmar being the poster child of their hate speech problem!
In 2022, Global Witness came back for one more look at Meta’s operations in Myanmar, this time with eight examples of real hate speech aimed at the Rohingya—actual posts from the period of the genocide, all taken from the UN Human Rights Council findings I’ve been linking to so frequently in this series. They submitted these real-life examples of hate speech to Meta as Burmese-language Facebook advertisements.
I promise you Meta doesn't give a shit about the $8 in ads its getting from Myanmar per year. This is more an issue of a giant corporation not noticing that some users in a foreign country speaking in a foreign language were doing horrible things with their platform. At worst its negligence. If it wasn't for the Facebook association most HN users would not even know Myanmar was in a civil war at all.
(Keep in mind after this, Facebook hired 20,000+ content moderators at great expense who they still employ to monitor for stuff like this).
According to the article, Facebook did make the deliberate decision not to dedicate resources to Myanmar and other non-western countries, so I don't think it's negligence. "Greed" is more appropriate.
Guy Rosen, Facebook’s VP of Integrity, told Sophie Zhang that the only coordinated fake networks Facebook would take down were the ones that affected the US, Western Europe, and “foreign adversaries.”
That's frankly sensible? There are three hundred+ countries on earth, to monitor all of their "fake networks" would take an organization like the CIAs. It's like saying its greed that Apple doesn't solve the family issues of all their workers. its not reasonable.
its greed that Apple doesn't solve the family issues of all their workers. its not reasonable.
Of course it's greed, and of course it's reasonable. Apple is directly responsible for the work conditions of its workers and the contractors they use, absolutely. If they can't be responsible then they can close shop. There is no obligation for them to have a business.
Maybe they should consider not operating in countries where they unmonitored presence may lead to genocide?
There's no obligation to operate everywhere. Instead, as discussed in previous parts, they invested heavily to get a monopoly in third world countries.
I have a hard time following this reasoning. I would definitely consider Facebook partly responsible for this genocide if they had deliberately made an editorial decision to spread the genocidal rhetoric.
But they didn't. The situation was as you put it mostly unmonitored. In other words, the spread of genocidal content was organic. This is the equivalent of blaming the phone company or the ISP for what the users do with the communication channels.
Yes, a history of trying to get to the truth of matters even if it's not popular in the zeitgeist.
Look at fsociety's comment in this same thread to see just how much misinformation is out there (which he helpfully corrects)
It just so happens a lot of misinformation is anti-corporate since many people want to believe every bad thing they hear about them. So I end up being one of the few commenters who say, actually, the big company did try here.
This is important because if every big company is supposed to be equally bad, there's no reputational penalty for the actually bad ones.
Funny, talking about the truth when your comment history is neck deep in right wing conspiracy and talking points.
The point people are making here is that Facebook shouldn't have done the impossible and created a perfect tech solution to moderate content, they should not have operated in these countries at all if they couldn't do it safely. They did not even try, instead they ignored internal reports on these problems for years and actively decided to make the problem worse with their single focus of growth. And for as you put it, only a few $ in advertisements, to fuel the fires of genocide.
Considering I've voted only democratic in every election in my life, and volunteered to help progressive candidates manage their campaigns in swing states, you're waaaay off base on my politics.
I really don't want to ask this since it run contrary to HN rules, but did you really read the article?
No one expect Facebook to remove all misinformation/hate speech/etc. That is unreasonable. But they have literally 1 data scientist to monitor the whole non-western world.
In a year where their annual profit is $23.9bn.
Assuming 1 data scientist is paid $500k/year, surely it is sensible to dedicate $20mn, less than 0.1% of their annual profit, to hire 40 more such data scientist and literally increase their effectiveness on this topic by 4000%? At the very least, this is what I expect from a company that actually have a speck of moral.
A strategic response manager told me that the world outside the US/Europe was basically like the wild west with me as the part-time dictator in my spare time. He considered that to be a positive development because to his knowledge it wasn’t covered by anyone before he learned of the work I was doing.
When you write “they have literally 1 data scientist to monitor the whole non-western world”, I assume you mean Sophie Zhang. I agree with a lot of what she said, and I mean a lot, but you have a misconception.
The integrity org, back when I was there, had thousands of folks and was aggressively hiring. They were not a lone island and were also supported by other orgs like data infra, FAIR, security, privacy, product teams and more.
Site integrity, Sophie’s team, was a small piece of the work done there. And they, like any team, relied heavily on other teams. You have all this unbelievable tooling internally, and internal teams are incentivized to get you to use it.
These issues were not from want of trying. The reality is that problems like this at scale are incredibly difficult. In my opinion, Frances trivialized a lot of this in her whistleblowing, but it makes a good news story so what can you do.
It pains me when people trivialize it as “just do the machine learnings to fix it all” or “if these tech companies actually gave a damn this wouldn’t be an issue”. It’s an incredibly hard problem, and much like security it will never be “solved”.
That doesn’t excuse when technology has a bad effect on the world. We need to be better, and for the most part as a society we are trying hard as hell. It’s also why the work is interesting and impactful. More should get into it.
I talked to FB people (recruiters, managers) in the site integrity team back in 2018. Admittedly I had some reservations up front, but during the on-site day I made up my mind to never even consider FB as a potential employer again. During a chat session between interviews, I asked about the prospect of doing proactive education - essentially, detecting users who had been caught in the influence operations and then surfacing them a note that they had been targeted by such activities, so that they could themselves make educated decisions.
The senior manager I was talking with at the time was visibly taken aback by the very idea. "We don't do that!"
From that experience I drew the inference that FB are fundamentally, as an organisation, incapable of doing the right thing. I suspect it's less about the cost, and more about the prospect of openly accepting accountability for what their platform is really used for.
I promise you Meta doesn't give a shit about the $8 in ads its getting from Myanmar per year.
They did certainly care about the growth of daily active users even if it meant they expanded to markets where they couldn't responsibly moderate their platform.
A company as big as Facebook is going to have different groups with different objectives that often do not see eye to eye or even know about each other. A company as profitable as Facebook is going to have projects that are motivated by things other than profits.
There is most likely a team at Facebook whose goal it is to get as many users as possible in developing markets. These markets do not generate profits from online ads. Facebook will never disclose its cost per user but in general online ads only make money from affluent users. I have heard that even mid income countries like Mexico or Romania are not profitable for ads. A place where people make an average of $3 a day is wildly unprofitable for Facebook and even the most optimistic development projections will not make them profitable for several decades. The OP is spot on that the amount of money that Facebook will ever receive from Myanmar is negligible.
Based on conversations with people at Facebook I generally get the sense that people there (especially at the top) have a strong sense of mission and a belief that their product makes the world a better place (I disagree with this but that doesn't change the fact). I would guess that these the project tasked with increasing users in developing countries is viewed more as philanthropy than business.
Facebook hired 20,000+ content moderators at great expense who they still employ to monitor for stuff like this
Part II of this series addresses this stat. During the peak genocidal years it seems there were only a handful of mods assigned to Myanmar, and none of them lived in Myanmar. To this day it sounds like there still may not be more than a hundred people moderating Myanmar content. Which is inline with the 20K number: 195 countries * 100 moderators per country = 19.5K mods total
I guess my main idea here is that when I look at the back-of-envelope numbers it's easy to see how even 100 moderators would have trouble keeping up with the content that millions of people produce. So you really do need those automated moderation tools to be effective.
Can someone please continue the thought experiment by estimating how much content each mod can review in a day, then extrapolating up to a week or month, and comparing to an estimate of how much content was actually posted on Facebook in the similar timeframe? I believe Part I or II provides an estimate like that.
Why "nowadays" ? In which century did big corporations behave responsibly ?
Of course Facebook didn't fix anything in Myanmar - the new junta government is both the one running the troll farms and responsible for the previous genocide !
Comments
So Facebook is presented with a trolley problem that have people lives on one track and more profit on the other track, and chose the latter, just as what we can expect from big corporations nowadays.
Not an unexpected decision overall, but it did surprise me how low Facebook can stoop. Like, they didn't even bother to fix their stuff in Myanmar after all these years with Myanmar being the poster child of their hate speech problem!
I promise you Meta doesn't give a shit about the $8 in ads its getting from Myanmar per year. This is more an issue of a giant corporation not noticing that some users in a foreign country speaking in a foreign language were doing horrible things with their platform. At worst its negligence. If it wasn't for the Facebook association most HN users would not even know Myanmar was in a civil war at all.
(Keep in mind after this, Facebook hired 20,000+ content moderators at great expense who they still employ to monitor for stuff like this).
According to the article, Facebook did make the deliberate decision not to dedicate resources to Myanmar and other non-western countries, so I don't think it's negligence. "Greed" is more appropriate.
That's frankly sensible? There are three hundred+ countries on earth, to monitor all of their "fake networks" would take an organization like the CIAs. It's like saying its greed that Apple doesn't solve the family issues of all their workers. its not reasonable.
If you can’t operate responsibly in 300+ markets then don’t operate in 300+ markets? Seems sensible to me.
If an airline doesn’t have enough pilots to service 20 routes they reduce their routes, not let passengers fly the plane.
What does operating responsibly mean? Everyone seems to have a different definition of that.
Then they shouldn't operate there if they can't do so without their users being harmed as a result
Like, block IP addresses?
Prevent creating accounts in countries they cannot operate responsibly. Disallow comments in languages they cannot moderate.
I promise you if they tried that people would complain even harder.
HN is never happy
Seems like a good start
Of course it's greed, and of course it's reasonable. Apple is directly responsible for the work conditions of its workers and the contractors they use, absolutely. If they can't be responsible then they can close shop. There is no obligation for them to have a business.
I wouldn’t think the CIA would be capable of doing such a thing and if it tried it would devolve into scandal one way or another.
Maybe they should consider not operating in countries where they unmonitored presence may lead to genocide?
There's no obligation to operate everywhere. Instead, as discussed in previous parts, they invested heavily to get a monopoly in third world countries.
I have a hard time following this reasoning. I would definitely consider Facebook partly responsible for this genocide if they had deliberately made an editorial decision to spread the genocidal rhetoric.
But they didn't. The situation was as you put it mostly unmonitored. In other words, the spread of genocidal content was organic. This is the equivalent of blaming the phone company or the ISP for what the users do with the communication channels.
This comment makes sense after reading his history.
Yes, a history of trying to get to the truth of matters even if it's not popular in the zeitgeist.
Look at fsociety's comment in this same thread to see just how much misinformation is out there (which he helpfully corrects)
It just so happens a lot of misinformation is anti-corporate since many people want to believe every bad thing they hear about them. So I end up being one of the few commenters who say, actually, the big company did try here.
This is important because if every big company is supposed to be equally bad, there's no reputational penalty for the actually bad ones.
Funny, talking about the truth when your comment history is neck deep in right wing conspiracy and talking points.
The point people are making here is that Facebook shouldn't have done the impossible and created a perfect tech solution to moderate content, they should not have operated in these countries at all if they couldn't do it safely. They did not even try, instead they ignored internal reports on these problems for years and actively decided to make the problem worse with their single focus of growth. And for as you put it, only a few $ in advertisements, to fuel the fires of genocide.
Considering I've voted only democratic in every election in my life, and volunteered to help progressive candidates manage their campaigns in swing states, you're waaaay off base on my politics.
I really don't want to ask this since it run contrary to HN rules, but did you really read the article?
No one expect Facebook to remove all misinformation/hate speech/etc. That is unreasonable. But they have literally 1 data scientist to monitor the whole non-western world.
In a year where their annual profit is $23.9bn.
Assuming 1 data scientist is paid $500k/year, surely it is sensible to dedicate $20mn, less than 0.1% of their annual profit, to hire 40 more such data scientist and literally increase their effectiveness on this topic by 4000%? At the very least, this is what I expect from a company that actually have a speck of moral.
When you write “they have literally 1 data scientist to monitor the whole non-western world”, I assume you mean Sophie Zhang. I agree with a lot of what she said, and I mean a lot, but you have a misconception.
The integrity org, back when I was there, had thousands of folks and was aggressively hiring. They were not a lone island and were also supported by other orgs like data infra, FAIR, security, privacy, product teams and more.
Site integrity, Sophie’s team, was a small piece of the work done there. And they, like any team, relied heavily on other teams. You have all this unbelievable tooling internally, and internal teams are incentivized to get you to use it.
These issues were not from want of trying. The reality is that problems like this at scale are incredibly difficult. In my opinion, Frances trivialized a lot of this in her whistleblowing, but it makes a good news story so what can you do.
It pains me when people trivialize it as “just do the machine learnings to fix it all” or “if these tech companies actually gave a damn this wouldn’t be an issue”. It’s an incredibly hard problem, and much like security it will never be “solved”.
That doesn’t excuse when technology has a bad effect on the world. We need to be better, and for the most part as a society we are trying hard as hell. It’s also why the work is interesting and impactful. More should get into it.
I'll provide a slightly different argument.
I talked to FB people (recruiters, managers) in the site integrity team back in 2018. Admittedly I had some reservations up front, but during the on-site day I made up my mind to never even consider FB as a potential employer again. During a chat session between interviews, I asked about the prospect of doing proactive education - essentially, detecting users who had been caught in the influence operations and then surfacing them a note that they had been targeted by such activities, so that they could themselves make educated decisions.
The senior manager I was talking with at the time was visibly taken aback by the very idea. "We don't do that!"
From that experience I drew the inference that FB are fundamentally, as an organisation, incapable of doing the right thing. I suspect it's less about the cost, and more about the prospect of openly accepting accountability for what their platform is really used for.
The series of articles makes it very clear that there are simple, non-high tech things Facebook could have done to mitigate the problem.
Insisting that it's hard and requires advanced ML is complete BS. Self delusion by FB'ers to give themselves and excuse for their complicity.
Purposely ignoring a problem is negligence.
They did certainly care about the growth of daily active users even if it meant they expanded to markets where they couldn't responsibly moderate their platform.
User growth and profits are not the same thing.
A company as big as Facebook is going to have different groups with different objectives that often do not see eye to eye or even know about each other. A company as profitable as Facebook is going to have projects that are motivated by things other than profits.
There is most likely a team at Facebook whose goal it is to get as many users as possible in developing markets. These markets do not generate profits from online ads. Facebook will never disclose its cost per user but in general online ads only make money from affluent users. I have heard that even mid income countries like Mexico or Romania are not profitable for ads. A place where people make an average of $3 a day is wildly unprofitable for Facebook and even the most optimistic development projections will not make them profitable for several decades. The OP is spot on that the amount of money that Facebook will ever receive from Myanmar is negligible.
Based on conversations with people at Facebook I generally get the sense that people there (especially at the top) have a strong sense of mission and a belief that their product makes the world a better place (I disagree with this but that doesn't change the fact). I would guess that these the project tasked with increasing users in developing countries is viewed more as philanthropy than business.
Part II of this series addresses this stat. During the peak genocidal years it seems there were only a handful of mods assigned to Myanmar, and none of them lived in Myanmar. To this day it sounds like there still may not be more than a hundred people moderating Myanmar content. Which is inline with the 20K number: 195 countries * 100 moderators per country = 19.5K mods total
I guess my main idea here is that when I look at the back-of-envelope numbers it's easy to see how even 100 moderators would have trouble keeping up with the content that millions of people produce. So you really do need those automated moderation tools to be effective.
Can someone please continue the thought experiment by estimating how much content each mod can review in a day, then extrapolating up to a week or month, and comparing to an estimate of how much content was actually posted on Facebook in the similar timeframe? I believe Part I or II provides an estimate like that.
Why "nowadays" ? In which century did big corporations behave responsibly ?
Of course Facebook didn't fix anything in Myanmar - the new junta government is both the one running the troll farms and responsible for the previous genocide !