Back when I was young, websites had this icon you could click that would take you straight to their RSS feed. You young whipper snappers have gone an fucked that up. Actually, I think it was Google's fault. When they killed their RSS reader people pronounced RSS dead so people just stopped publishing RSS feeds or just didn't link to them.
* Yes, I know the article talks about the RSS icon, i'm just soapboxing.
Even better, for a few months the browsers themselves would highlight RSS feeds and allow you subscribe right in the browser. It was too good to last.
RSS is great but it has one great flaw in that it doesn't scale that well by itself. If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS.
What does work well (and how things have evolved) is to have a service that polls RSS on behalf of its users. This was the beauty of Google Reader but plenty of replacements exist.
RSS is great but it has one great flaw in that it doesn't scale that well by itself
The readers often cache the result for all users of the same feed. The only exception are readers that are running 100% locally. In that case using Etag or Last-Modified will make each request cheap and manageble.
That is my point. RSS by itself is a scaling problem but online readers make it manageable by polling sites on a reasonable rate and redistributing new articles to their users.
RSS is great but it has one great flaw in that it doesn't scale that well by itself. If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS.
Who's updating their RSS feeds once an hour 24 hours a day?
Anyway, a very popular site with that many millions of visitors already has to handle extreme traffic, regardless of RSS.
“Who's updating their RSS feeds once an hour 24 hours a day?”
RSS is a pull-type system, no?. So the end-user is causing the overload problem by hitting the publisher’s RSS feed every hour? The problem is out of the hands of the publisher…
So the end-user is causing the overload problem by hitting the publisher’s RSS feed every hour?
There is no overload problem.
And again, this math is off, because an end user is not even awake 24 hours a day: "If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS."
"...an end user is not even awake 24 hours a day..."
Sorry, I wasn't thinking about the end user literally pushing a button. I was reading into the discussion here what I imagined to be a gap in the overall design philosophy of RSS where the end-user, using their own app or preferred methods, poll all of their favorite websites selecting among common defaults (Select the frequency you like to poll this RSS feed: weekly? daily? hourly? hehe minutely? secondly?).
In this scenario the user doesn't need to be awake 24 hours a day. They just need to use software which stupidly hits someone's server like they're doing a denial of service attack. As your comment alludes no one needs to be that up-to-date.
In contrast, there was another solution: "...service that polls RSS on behalf of its users."
Again, reading the discussion I imagined the wild west of the internet has inexpert users choosing unsane defaults and overloading small self-hosting content providers. Compared to third-party aggregators who are trying to make money by solving a "Tragedy of the Commons" problem.
In this scenario the user doesn't need to be awake 24 hours a day. They just need to use software which naively hits someone's server like they're doing a denial of service attack.
But the user's computer would need to be awake 24 hours a day, which is usually not the case.
Is that wasteful? Does that make me a poor Internet citizen?
I personally don't think it's a problem. As I said, "a very popular site with that many millions of visitors already has to handle extreme traffic, regardless of RSS."
I'd never considered using RSS feeds from an app and pulling it directly. I have too many devices for that to be a great workflow.
A lot of apps have sync to handle that workflow.
my gut feeling on this is that the people still using RSS and people self-hosting would overlap in a big way.
My gut feeling is that most RSS users, including myself, simply use client apps.
A $160 n100 minipc should have no problem serving an essentially static file out of nginx at well above that rate. If you use proper caching headers and don't publish new items that often, it should basically be idle.
Feedly has relatively adequate RSS builder based on site elements for sites without RSS, but you only get a few feeds with pro plan. Wish there was open implementation. Also seems like something LLM would excel at.
I'm still continually astounded that the tech community appeared to absolutely swallow Google's utter bullshit answers about why they shut down Reader; that really marked a turning point.
Comments
Back when I was young, websites had this icon you could click that would take you straight to their RSS feed. You young whipper snappers have gone an fucked that up. Actually, I think it was Google's fault. When they killed their RSS reader people pronounced RSS dead so people just stopped publishing RSS feeds or just didn't link to them.
* Yes, I know the article talks about the RSS icon, i'm just soapboxing.
Even better, for a few months the browsers themselves would highlight RSS feeds and allow you subscribe right in the browser. It was too good to last.
RSS is great but it has one great flaw in that it doesn't scale that well by itself. If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS.
What does work well (and how things have evolved) is to have a service that polls RSS on behalf of its users. This was the beauty of Google Reader but plenty of replacements exist.
The readers often cache the result for all users of the same feed. The only exception are readers that are running 100% locally. In that case using Etag or Last-Modified will make each request cheap and manageble.
That is my point. RSS by itself is a scaling problem but online readers make it manageable by polling sites on a reasonable rate and redistributing new articles to their users.
I use the Awesome RSS add-on for that in Firefox: https://addons.mozilla.org/en-US/firefox/addon/awesome-rss/
Firefox removing RSS support was their capitulation in supporting the open web.
That is very ambiguous phrasing.
Firefox removing RSS support was the nail in the coffin for their support of the Open Web.
How long after Google Reader was canceled did FF remove RSS?
Google shut down Google Reader on July 1, 2013.
Firefox removed RSS support in version 64 on December 11, 2018, approximately 5 years and 5 months later.
Thank you! I would have never looked that up.
Who's updating their RSS feeds once an hour 24 hours a day?
Anyway, a very popular site with that many millions of visitors already has to handle extreme traffic, regardless of RSS.
I think Miniflux polls every hour by default. Not sure about others.
According to my logs, some feed readers poll every 10 minutes or so.
“Who's updating their RSS feeds once an hour 24 hours a day?”
RSS is a pull-type system, no?. So the end-user is causing the overload problem by hitting the publisher’s RSS feed every hour? The problem is out of the hands of the publisher…
Yes.
The web in general is also a pull-type system.
There is no overload problem.
And again, this math is off, because an end user is not even awake 24 hours a day: "If 2 million people subscribe to your feed and try to update it once an hour, that is 48 million requests a day just for RSS."
Sorry, I wasn't thinking about the end user literally pushing a button. I was reading into the discussion here what I imagined to be a gap in the overall design philosophy of RSS where the end-user, using their own app or preferred methods, poll all of their favorite websites selecting among common defaults (Select the frequency you like to poll this RSS feed: weekly? daily? hourly? hehe minutely? secondly?).
In this scenario the user doesn't need to be awake 24 hours a day. They just need to use software which stupidly hits someone's server like they're doing a denial of service attack. As your comment alludes no one needs to be that up-to-date.
In contrast, there was another solution: "...service that polls RSS on behalf of its users."
Again, reading the discussion I imagined the wild west of the internet has inexpert users choosing unsane defaults and overloading small self-hosting content providers. Compared to third-party aggregators who are trying to make money by solving a "Tragedy of the Commons" problem.
But the user's computer would need to be awake 24 hours a day, which is usually not the case.
And once per hour is not a DoS attack.
My home server is awake 24 hours a day.
Is that wasteful? Does that make me a poor Internet citizen?
I'd never considered using RSS feeds from an app and pulling it directly. I have too many devices for that to be a great workflow.
Normally I'd say I was an outlier, but my gut feeling on this is that the people still using RSS and people self-hosting would overlap in a big way.
I personally don't think it's a problem. As I said, "a very popular site with that many millions of visitors already has to handle extreme traffic, regardless of RSS."
A lot of apps have sync to handle that workflow.
My gut feeling is that most RSS users, including myself, simply use client apps.
Surely this is solved by putting RSS behind Cloudflare.
Sure, it's solved by Cloudflare breaking the RSS feed entirely, which often happens when people put their websites behind Cloudflare.
I think it should be noted that this happens when not configuring things right.
By default, security rules are on, but I can disable security rules programmatically for a hostname too.
fwiw, I once got a ddos on a host running their pages product, and got no charge for it. it also stayed up and didn't give captcha pages
Vivaldi browser still does that.
A $160 n100 minipc should have no problem serving an essentially static file out of nginx at well above that rate. If you use proper caching headers and don't publish new items that often, it should basically be idle.
A lot of clients don't respect caching headers[0], but yeah, static content hosting is a solved problem.
And there are always CDNs if you really have a large, global audience.
[0] http://rachelbythebay.com/w/2024/10/25/fs/
That's exactly why pubsubhubbub, now websub, has been created. Around 10 years ago.
But that's just a Cloudflare problem, no?
Feedly has relatively adequate RSS builder based on site elements for sites without RSS, but you only get a few feeds with pro plan. Wish there was open implementation. Also seems like something LLM would excel at.
I was about to say, that was all Google.
I'm still continually astounded that the tech community appeared to absolutely swallow Google's utter bullshit answers about why they shut down Reader; that really marked a turning point.