So, immutability and/or api contract is important here.
If I'm pulling in a function, I want it to do what I think I want. Sometimes I want that to change (get a bug fix), but sometimes I don't (someone introduces a bug, or makes the func more general and introduces slowdowns).
This feels like a job for a content-addressable git-like tool. How about this:
I can discover my function (via whatever means). The function is actually named 8804ea505fda087da53b799434c377f015933707 (the sha-something of it's (normalised?) textual representation).
I then import it into my codebase as "useful_fun". My code reads like:
useful_fun("do it", "to it")
but I have some kind of dependencies/import record which says that "useful_fun" is actually 8804ea505fda087da53b799434c377f015933707. That means one and only one thing across all time, the func with that hash.
So how do we handle updates? If we want a golang-like model, the developer could run something like "update deps". This would:
- go back to the central repository, looking for updates to 8804ea505fda087da53b799434c377f015933707. It might find 5. Local policy then determines what happens. Could be "always choose the original authors update" or "choose the one with the most votes" or "always ask the dev, showing diffs".
Note that because the unique name is based on the function content, any change to it creates a new item in the db. (Content-addressability, same way git and other systems do it.)
- stuff can be grouped and batched. If I pull in 10 functions tagged with the same project ('module') and they've all been updated, I can say "and do the same with all the others".
- This kind of metadata allows all kind of good stuff. I can subscribe to alerts on the functions I've imported and get told about new versions, or security warnings. This kind of subscription information can be used as a popularity contest to solve the "which fork on github do I want to use" problem?
- people can still publish modules. They now look like a git directory or tree. A git tree is a blob which contains the hashes of the files within it. A 'module' could be a blob which specifies which (immutable) functions are in it.
If we use normalised functions, we've now got a module representation which allows arbitrary functions to be pulled together. At fetch time, we can denormalise into the user's preferred coding style. At push time, we renormalise. We aren't grouping stuff into files, so a 'project' or a 'module' consists solely of the semantic contents, nothing to do with artificial grouping for the file system.
I think Mr. Armstrong would approve, given his comments near the end of https://www.youtube.com/watch?v=lKXe3HUG2l4, where he opines that the web would be great if, instead of URLs, every published document were just named with a hash of its content.
every published document were just named with a hash of its content.
I see too many issues with this (for example):
- I publish a news article. I publish a retraction/update to said article. Now the article has a new hash. Does the old hash give you the old version of the article, or redirect you to the new version?
- How do we define 'document?' If we define it as the complete HTML page served up to the browser, then changes to the design of the site would invalidate all previous hashes. Pointing old hashes to new hashes is work, which will not always be done (leading to the same situation we have with site redesigns breaking old URLs).
http://ipfs.io lets you to reference content in one of two ways. Either an immutable hash of the content or a reference to the public key that's allowed to publish / update an immutable hash of the content. Seems like a pretty good compromise.
Exactly, you should still be able to have references to persistent identities. Much like the semantics of clojure which has a distinction between values and references to identities like vars/agents etc.
Why not keep the old document and let the new version (child) refer to the old one (parent)? You then "just" need a refresh feature that can retrieve newer versions of the document for you. In our P2Pedia system (I referred to it in a sibling post earlier) you can go from the parent to its children via search.
We have a P2P file-sharing program that does this, called U-P2P (http://u-p2p.sf.net). Content is hashed, and you use a Gnutella search using the hash to retrieve it. Documents are organized by what we call "communities", which themselves are represented by a document and its corresponding hash. So the document name is really made up of two hashes: the one of the community it belongs to, and its own hash. You can use these hashes as hyperlinks, and U-P2P resolves it via search, as previously mentioned.
What we think is great about it is that the hash is location-independent. There could be multiple copies of the document at various locations at any given point. As long as there is at least one copy and that it is reachable via search, it will be retrieved.
We also built a distributed Wiki based on that idea and platform, called P2Pedia (http://p2pedia.sf.net).
It's all very much an academic research project, so don't expect a beautiful interface or easy-to-install packaging or anything, but I think it's a good proof of concept.
(note to self: we should really move these to GitHub).
In React.js, you can serialize your whole app state through a simple ˋJSON.stringify` and base64 encode that into the url. The nice property of that is that you get to pass that url around to friends, and when they click on it they'll go to the page, which decodes and deserializes the url and reproduce the exact app state, down to the letters in the input boxes.
Effectively, this gives you "program as a value" where the same url means the same program. Immutable programs basically.
I've tried this and the current downside is that it looks extremely ugly when you try to share a link lol. But this should be circumventable. The other downside is that this is a bit theoretical still. You'll have to exclude sensitive information such as password. Sometimes stuff are in a closure rather than in your ˋstate`.
It's initially just doing the simplest possible thing (making the resource unavailable unless its hash is valid) but semantically it will probably be allowed for the browser to resolve the resource using other methods (e.g. if it already has that resource cached from another URL) so long as the hash matches.
So we could simply set up a url shortening service that published such hashes. Unfortunately with the 'dynamic' nature of web pages these days that's going to be hard to go back to. It may be an interesting way to re-boot the web though. 'regular' Urls are then merely a DNS like layer on top of a content hashing scheme.
Comments
So, immutability and/or api contract is important here.
If I'm pulling in a function, I want it to do what I think I want. Sometimes I want that to change (get a bug fix), but sometimes I don't (someone introduces a bug, or makes the func more general and introduces slowdowns).
This feels like a job for a content-addressable git-like tool. How about this:
I can discover my function (via whatever means). The function is actually named 8804ea505fda087da53b799434c377f015933707 (the sha-something of it's (normalised?) textual representation).
I then import it into my codebase as "useful_fun". My code reads like:
but I have some kind of dependencies/import record which says that "useful_fun" is actually 8804ea505fda087da53b799434c377f015933707. That means one and only one thing across all time, the func with that hash.So how do we handle updates? If we want a golang-like model, the developer could run something like "update deps". This would:
- go back to the central repository, looking for updates to 8804ea505fda087da53b799434c377f015933707. It might find 5. Local policy then determines what happens. Could be "always choose the original authors update" or "choose the one with the most votes" or "always ask the dev, showing diffs".
Note that because the unique name is based on the function content, any change to it creates a new item in the db. (Content-addressability, same way git and other systems do it.)
- stuff can be grouped and batched. If I pull in 10 functions tagged with the same project ('module') and they've all been updated, I can say "and do the same with all the others".
- This kind of metadata allows all kind of good stuff. I can subscribe to alerts on the functions I've imported and get told about new versions, or security warnings. This kind of subscription information can be used as a popularity contest to solve the "which fork on github do I want to use" problem?
- people can still publish modules. They now look like a git directory or tree. A git tree is a blob which contains the hashes of the files within it. A 'module' could be a blob which specifies which (immutable) functions are in it.
If we use normalised functions, we've now got a module representation which allows arbitrary functions to be pulled together. At fetch time, we can denormalise into the user's preferred coding style. At push time, we renormalise. We aren't grouping stuff into files, so a 'project' or a 'module' consists solely of the semantic contents, nothing to do with artificial grouping for the file system.
Seems like an interesting future.
I think Mr. Armstrong would approve, given his comments near the end of https://www.youtube.com/watch?v=lKXe3HUG2l4, where he opines that the web would be great if, instead of URLs, every published document were just named with a hash of its content.
I see too many issues with this (for example):
- I publish a news article. I publish a retraction/update to said article. Now the article has a new hash. Does the old hash give you the old version of the article, or redirect you to the new version?
- How do we define 'document?' If we define it as the complete HTML page served up to the browser, then changes to the design of the site would invalidate all previous hashes. Pointing old hashes to new hashes is work, which will not always be done (leading to the same situation we have with site redesigns breaking old URLs).
http://ipfs.io lets you to reference content in one of two ways. Either an immutable hash of the content or a reference to the public key that's allowed to publish / update an immutable hash of the content. Seems like a pretty good compromise.
Exactly, you should still be able to have references to persistent identities. Much like the semantics of clojure which has a distinction between values and references to identities like vars/agents etc.
These URLs would be clearly marked of course.
Why not just have all URLs be mutable aliases for hashes?
Why not keep the old document and let the new version (child) refer to the old one (parent)? You then "just" need a refresh feature that can retrieve newer versions of the document for you. In our P2Pedia system (I referred to it in a sibling post earlier) you can go from the parent to its children via search.
We have a P2P file-sharing program that does this, called U-P2P (http://u-p2p.sf.net). Content is hashed, and you use a Gnutella search using the hash to retrieve it. Documents are organized by what we call "communities", which themselves are represented by a document and its corresponding hash. So the document name is really made up of two hashes: the one of the community it belongs to, and its own hash. You can use these hashes as hyperlinks, and U-P2P resolves it via search, as previously mentioned.
What we think is great about it is that the hash is location-independent. There could be multiple copies of the document at various locations at any given point. As long as there is at least one copy and that it is reachable via search, it will be retrieved.
We also built a distributed Wiki based on that idea and platform, called P2Pedia (http://p2pedia.sf.net).
It's all very much an academic research project, so don't expect a beautiful interface or easy-to-install packaging or anything, but I think it's a good proof of concept.
(note to self: we should really move these to GitHub).
In React.js, you can serialize your whole app state through a simple ˋJSON.stringify` and base64 encode that into the url. The nice property of that is that you get to pass that url around to friends, and when they click on it they'll go to the page, which decodes and deserializes the url and reproduce the exact app state, down to the letters in the input boxes.
Effectively, this gives you "program as a value" where the same url means the same program. Immutable programs basically.
I've tried this and the current downside is that it looks extremely ugly when you try to share a link lol. But this should be circumventable. The other downside is that this is a bit theoretical still. You'll have to exclude sensitive information such as password. Sometimes stuff are in a closure rather than in your ˋstate`.
And the other other downside would be when your app becomes big enough to not fit in an url
Emerging standard in that area, subresource integrity: http://w3c.github.io/webappsec/specs/subresourceintegrity/
It's initially just doing the simplest possible thing (making the resource unavailable unless its hash is valid) but semantically it will probably be allowed for the browser to resolve the resource using other methods (e.g. if it already has that resource cached from another URL) so long as the hash matches.
So we could simply set up a url shortening service that published such hashes. Unfortunately with the 'dynamic' nature of web pages these days that's going to be hard to go back to. It may be an interesting way to re-boot the web though. 'regular' Urls are then merely a DNS like layer on top of a content hashing scheme.