The blog mentions the "discovery problem" 7 times but this project's particular technology architecture for syndication doesn't seem to actually address that.
The project's main differentiating factor seems to be not propagating the actual content to the nodes but instead save disk space by only distributing hashes of content.
However, having a "p2p" decentralized network of hashes doesn't solve the "discovery" problem. The blog lists the following bullet points of metadata but that's not enough to facilitate "content discovery":
>However it could be possible to build a scalable and fast decentralized infrastructure if instances only kept references to hosted content.
Let’s define what could be the absolute minimum structure of decentralized content unit:
- Reference to your content — a URL
- User ID — A way to identify who posted the content (domain name)
- Signature — A way to verify that the user is the actual owner
- Content hash — A way to identify if content was changed after publishing
- Post time — A way to know when the post was submitted to the platform
It is not unreasonable to expect that all this information could fit into roughly 100 bytes.
Those minimal 5 fields of metadata (url+userid+sig+hash+time) are not enough to facilitate content discovery.
Content discovery of reducing the infinite internet down to a manageable subset requires a lot more metadata. That extra metadata requires scanning the actual content instead of the hashes. This extra metadata based on actual content (e.g. Google's "search index", Twitter's tweets & hashtags, etc) -- is one of the factors that acts as unescapable gravity pulling users towards centralization.
To the author, what algorithm did you have in mind for decentralized content discovery?
Thanks for the comment, these concerns are valid. At the core the protocol supports only basic discovery - you can see who is posting right now and history of everyone who has ever posted. Regarding rich context discovery where content could be found by specific tags and key words this would be implemented by reader platforms that crawl the index
Comments
The blog mentions the "discovery problem" 7 times but this project's particular technology architecture for syndication doesn't seem to actually address that.
The project's main differentiating factor seems to be not propagating the actual content to the nodes but instead save disk space by only distributing hashes of content.
However, having a "p2p" decentralized network of hashes doesn't solve the "discovery" problem. The blog lists the following bullet points of metadata but that's not enough to facilitate "content discovery":
>However it could be possible to build a scalable and fast decentralized infrastructure if instances only kept references to hosted content.
Those minimal 5 fields of metadata (url+userid+sig+hash+time) are not enough to facilitate content discovery.
Content discovery of reducing the infinite internet down to a manageable subset requires a lot more metadata. That extra metadata requires scanning the actual content instead of the hashes. This extra metadata based on actual content (e.g. Google's "search index", Twitter's tweets & hashtags, etc) -- is one of the factors that acts as unescapable gravity pulling users towards centralization.
To the author, what algorithm did you have in mind for decentralized content discovery?
Thanks for the comment, these concerns are valid. At the core the protocol supports only basic discovery - you can see who is posting right now and history of everyone who has ever posted. Regarding rich context discovery where content could be found by specific tags and key words this would be implemented by reader platforms that crawl the index