But people are saying that the Fediverse could replace Twitter and Facebook and Tiktok and Instagram and what not, don't they?
How much hardware would you need for 100 million MAUs? (And that's just a fraction of the current social media users.)
If it really "scales" like indicated in the parent post this tech will never provide any alternative to centralized social media sites just for technical reasons no matter what people want or do.
Maybe someone experienced in effective distributed systems should start to design an alternative.
Otherwise there won't be any viable alternative to the commercial silos no matter how bad people would like one.
Mastodon absolutely does not scale that far and this should be considered fact.
Far smaller instance owners sometimes complain about bills of hundreds of USD per month.
So the only way to scale up "indefinitely" is by having many small/medium-sized instances, but not really. In a 100M+ network, instances will suffer due to the wasteful nature of federation plus social media being append-only.
Costs will forever go up and this doesn't even mention the burden and liabilities of moderation.
Amidst all our hate for big social, we've forgotten about all the things they do very well. They are (financially) free. They are reliable and scale up without you noticing. You do not have to generally worry about your entire account and content being gone because some mod gave up. If you don't do anything funny, all your content is preserved, forever. Moderation works reasonably well, even if never perfect.
We've taken all that for granted. But it costs billions, an army of engineers, mods, legal, marketing, UX, top notch infrastructure to make it run and work this smoothly.
The idea that a bunch of enthusiasts can replicate this, is misplaced.
Yet, my 2011 Google+ post is not still there, so this seems like a crap shoot: your argument is inconsistent at best, but frankly feels more like you've bought into a lie.
Costs will forever go up and this doesn't even mention the burden and liabilities of moderation.
Storage costs have historically dropped so fast that this just isn't an issue I'd worry about. I ran a mail provider with ~2m accounts around 2000. We had less aggregate storage than my laptop has on a single M.2 SSD now. With redundancy, the cost per GB for us at that time was around 10,000 times higher than it'd have been today. Our total processing power was lower than my laptop. Our bandwidth use was a tiny fraction of my home internet connection.
In other words: If it's an issue today, it soon won't be. Costs are low enough per user today to be viable, and they'll only drop.
You do not have to generally worry about your entire account and content being gone because some mod gave up.
I do have to worry about what happens when I have to start again because engagement is cratering and I can't just migrate elsewhere or run my own, though (e.g. I get more engagement on Mastodon than on Twitter despite 100x as many followers on one of my Twitter accounts). Since I can (and did) choose to run my own instance, I don't need to worry about that again.
The idea that a bunch of enthusiasts can replicate this, is misplaced.
This was the kind of argument used against open source 20-30 years ago. It's was just as ridiculous an argument then. This is magnitudes easier than what was achieved with open source, and it's also flawed because just as detractors of open source you're presuming that it will only be enthusiasts, and that e.g. nobody will start and operate commercial services for those who prefer that (some commercial hosters already exists for Mastodon for example).
I don't buy the "storage is cheap, bandwidth is cheap" argument.
The Fediverse is much like binary internet news. Something that is to this day not really operable for reasonable cost, especially not for small or even private providers.
A few hundred users make it necessary to shovel around TBs of data. For every involved server. This just doesn't scale, no matter how cheap bandwidth and storage will get in the next years.
You can not buy it all you want, but it's true. Consider that even if you were right that it takes only a few hundred users to require shoveling TB's of data (it's not, assuming we're talking monthly, unless you happen to be hosting e.g. George Takei and similar), the ~$5/month VPS I run my instance on can do that just fine at no extra cost. It's not necessary to pay insane AWS egress rates.
For every involved server.
This is misleading. Not every server sees anywhere close to every post, because most servers are not "involved" with any given user, and that becomes increasingly true the larger the network becomes. Larger servers sees a large proportion but also receives only one copy of each post even though they're likely to have multiple recipients.
Realistically, the number we're looking at for the source server is about half the total network cost for that set of users.
This just doesn't scale, no matter how cheap bandwidth and storage will get in the next years.
The cost per user of running a Fediverse instance is dropping, not rising, even as the network size has exploded. And as it is, the main cost of running e.g. Mastodon is that Mastodon, fittingly, is a huge, inefficient, resource-hungry beast. Not that federation is inherently costly.
My prediction is that the average cost per user for bandwidth, compute, and storage will be <$0.10/month within a couple of years. It's probably below that already if done efficiently enough at reasonable scale.
The bulk of the cost will be admin, moderation and ops, which scales linearly or sub-linearly with number of users.
Realistically, the number we're looking at for the source server is about half the total network cost for that set of users.
Now think again what that actually means.
My prediction is that the average cost per user for bandwidth, compute, and storage will be <$0.10/month within a couple of years.
So you think it's much more expensive at the moment, right?
But $0.10/month means already $100/month just for someone who has 1000 "friends". And that's not a big amount. Regular people on FB have often collected more "friends"…
A YouTube account with 10 000 followers is actually quite small. But that would be already 1 000 bucks! Most people on this planet don't earn so much money in a whole month. And this would be just the cost for some smallish hobby account…
So even when we take your numbers, it's extremely expensive! How is this going to scale beyond some privileged geeks in the western world?
I see absolutely nothing about ActivityPub that is inherently hard to scale. Adding more points of presence & smart fan-out seems like it could keep scaling indefinitely, from what I can see.
Costs will forever go up
Let's simplify follower-ship and assume instead a simpler model of bi-directional friending. If you ahve M users each of which have N total friends, the naive fear is this: that costs will keep growing terrifyingly. But I expect it's actually more a sigmoid curve. As you grow into millions and especially many-millions of users, more and more you'll have your people following the same person - which reduces traffic - and more and more of your people followed by multiple people on another server - which reduces traffic. The actual growth curve here is more sigmoidal, with a big variability depending on the number of active instances fediverse instances out there.
You do not have to generally worry about your entire account and content being gone because some mod gave up.
To get listed on joinmastadon you have to promise to give at least 3 months heads-up, during which time users have to be able to activate an account transfer to other systems. There was a case where mastodon.au nearly pulled the plug with a short notice, but someone else stepped up to take over the system.
The potential for this to be a huge problem is absolutely super real, but in practice, I have almost no real concern over this and think it should be broadly disregarded. The server I was on shut down, but the mod gave us almost a full year of continued service, and another year of read-only service during which we can transfer. Trying to drum up fear & doubt over the instability of this system seems irresponsible & premature, given how well things have gone.
The idea that a bunch of enthusiasts can replicate this, is misplaced.
I agree that there are huge advantages & I think we absolutely have taken much for granted. Right now you're looking at this through shit-colored glasses though, and I think it's naive to wish so hard for this all to fail. It's convenient & easy to say it'll never work.
But there's so many strategies where we can start to turn the scale into something useful. If servers publish WebBundles of user's feeds, users can p2p distribute the signed http content among themselves, perhaps via WebTorrent. Rather than live in fear of our users and our scaling, we can rely on our users to help us tackle scaling. Yes we can, man. It's totally doable. Get off the ledge.
"and I think it's naive to wish so hard for this all to fail. It's convenient & easy to say it'll never work."
I do not wish for it to fail, I'm observing that it cannot succeed within the scope of our discussion, which was Mastodon as a "big social" replacement. We're talking 0.5-2B MAU.
Comments
But people are saying that the Fediverse could replace Twitter and Facebook and Tiktok and Instagram and what not, don't they?
How much hardware would you need for 100 million MAUs? (And that's just a fraction of the current social media users.)
If it really "scales" like indicated in the parent post this tech will never provide any alternative to centralized social media sites just for technical reasons no matter what people want or do.
Maybe someone experienced in effective distributed systems should start to design an alternative.
Otherwise there won't be any viable alternative to the commercial silos no matter how bad people would like one.
Mastodon absolutely does not scale that far and this should be considered fact. Far smaller instance owners sometimes complain about bills of hundreds of USD per month.
So the only way to scale up "indefinitely" is by having many small/medium-sized instances, but not really. In a 100M+ network, instances will suffer due to the wasteful nature of federation plus social media being append-only.
Costs will forever go up and this doesn't even mention the burden and liabilities of moderation.
Amidst all our hate for big social, we've forgotten about all the things they do very well. They are (financially) free. They are reliable and scale up without you noticing. You do not have to generally worry about your entire account and content being gone because some mod gave up. If you don't do anything funny, all your content is preserved, forever. Moderation works reasonably well, even if never perfect.
We've taken all that for granted. But it costs billions, an army of engineers, mods, legal, marketing, UX, top notch infrastructure to make it run and work this smoothly.
The idea that a bunch of enthusiasts can replicate this, is misplaced.
Are you really not able to think of any dead social networks that deleted all of their content? Hell: Google alone has managed to delete at least two!
Obviously with forever I mean within the lifespan of the service, not literally forever as in a trillion years.
So yes, your 2008 Facebook post is still going to be there.
Yet, my 2011 Google+ post is not still there, so this seems like a crap shoot: your argument is inconsistent at best, but frankly feels more like you've bought into a lie.
Storage costs have historically dropped so fast that this just isn't an issue I'd worry about. I ran a mail provider with ~2m accounts around 2000. We had less aggregate storage than my laptop has on a single M.2 SSD now. With redundancy, the cost per GB for us at that time was around 10,000 times higher than it'd have been today. Our total processing power was lower than my laptop. Our bandwidth use was a tiny fraction of my home internet connection.
In other words: If it's an issue today, it soon won't be. Costs are low enough per user today to be viable, and they'll only drop.
I do have to worry about what happens when I have to start again because engagement is cratering and I can't just migrate elsewhere or run my own, though (e.g. I get more engagement on Mastodon than on Twitter despite 100x as many followers on one of my Twitter accounts). Since I can (and did) choose to run my own instance, I don't need to worry about that again.
This was the kind of argument used against open source 20-30 years ago. It's was just as ridiculous an argument then. This is magnitudes easier than what was achieved with open source, and it's also flawed because just as detractors of open source you're presuming that it will only be enthusiasts, and that e.g. nobody will start and operate commercial services for those who prefer that (some commercial hosters already exists for Mastodon for example).
I don't buy the "storage is cheap, bandwidth is cheap" argument.
The Fediverse is much like binary internet news. Something that is to this day not really operable for reasonable cost, especially not for small or even private providers.
A few hundred users make it necessary to shovel around TBs of data. For every involved server. This just doesn't scale, no matter how cheap bandwidth and storage will get in the next years.
You can not buy it all you want, but it's true. Consider that even if you were right that it takes only a few hundred users to require shoveling TB's of data (it's not, assuming we're talking monthly, unless you happen to be hosting e.g. George Takei and similar), the ~$5/month VPS I run my instance on can do that just fine at no extra cost. It's not necessary to pay insane AWS egress rates.
This is misleading. Not every server sees anywhere close to every post, because most servers are not "involved" with any given user, and that becomes increasingly true the larger the network becomes. Larger servers sees a large proportion but also receives only one copy of each post even though they're likely to have multiple recipients.
Realistically, the number we're looking at for the source server is about half the total network cost for that set of users.
The cost per user of running a Fediverse instance is dropping, not rising, even as the network size has exploded. And as it is, the main cost of running e.g. Mastodon is that Mastodon, fittingly, is a huge, inefficient, resource-hungry beast. Not that federation is inherently costly.
My prediction is that the average cost per user for bandwidth, compute, and storage will be <$0.10/month within a couple of years. It's probably below that already if done efficiently enough at reasonable scale.
The bulk of the cost will be admin, moderation and ops, which scales linearly or sub-linearly with number of users.
Now think again what that actually means.
So you think it's much more expensive at the moment, right?
But $0.10/month means already $100/month just for someone who has 1000 "friends". And that's not a big amount. Regular people on FB have often collected more "friends"…
A YouTube account with 10 000 followers is actually quite small. But that would be already 1 000 bucks! Most people on this planet don't earn so much money in a whole month. And this would be just the cost for some smallish hobby account…
So even when we take your numbers, it's extremely expensive! How is this going to scale beyond some privileged geeks in the western world?
Maybe?
I see absolutely nothing about ActivityPub that is inherently hard to scale. Adding more points of presence & smart fan-out seems like it could keep scaling indefinitely, from what I can see.
Let's simplify follower-ship and assume instead a simpler model of bi-directional friending. If you ahve M users each of which have N total friends, the naive fear is this: that costs will keep growing terrifyingly. But I expect it's actually more a sigmoid curve. As you grow into millions and especially many-millions of users, more and more you'll have your people following the same person - which reduces traffic - and more and more of your people followed by multiple people on another server - which reduces traffic. The actual growth curve here is more sigmoidal, with a big variability depending on the number of active instances fediverse instances out there.
To get listed on joinmastadon you have to promise to give at least 3 months heads-up, during which time users have to be able to activate an account transfer to other systems. There was a case where mastodon.au nearly pulled the plug with a short notice, but someone else stepped up to take over the system.
The potential for this to be a huge problem is absolutely super real, but in practice, I have almost no real concern over this and think it should be broadly disregarded. The server I was on shut down, but the mod gave us almost a full year of continued service, and another year of read-only service during which we can transfer. Trying to drum up fear & doubt over the instability of this system seems irresponsible & premature, given how well things have gone.
I agree that there are huge advantages & I think we absolutely have taken much for granted. Right now you're looking at this through shit-colored glasses though, and I think it's naive to wish so hard for this all to fail. It's convenient & easy to say it'll never work.
But there's so many strategies where we can start to turn the scale into something useful. If servers publish WebBundles of user's feeds, users can p2p distribute the signed http content among themselves, perhaps via WebTorrent. Rather than live in fear of our users and our scaling, we can rely on our users to help us tackle scaling. Yes we can, man. It's totally doable. Get off the ledge.
"and I think it's naive to wish so hard for this all to fail. It's convenient & easy to say it'll never work."
I do not wish for it to fail, I'm observing that it cannot succeed within the scope of our discussion, which was Mastodon as a "big social" replacement. We're talking 0.5-2B MAU.
Given an informed yardstick of $.02-.05 per MAU for a 10k instance, possibly $2-5M/month, though much depends on the profile of users per instance.
The support, moderation, legal, administration, and corporate costs would dwarf this cloud/hardware cost.