If you are going to use HTTPS, you need to use it for all content on that domain. Otherwise if you load for example a large javascript file over HTTP, the attacker can just poison that file and control your whole page.
Even if you loaded an image from the same domain, your credentials would sent sent as a cookie in plain text.
The web is long overdue for a method to specify an exact resource, by content-hash, from one-of-whatever-sources.
Those sources can then be other less-secure protocols, even those unanticipated by the referrer, because the client got the necessary verifier via the secure-path.
It would need to be done via the pointer-to-content (URL/URI), and an independently-calculated secure-hash, not just a header. (The 'ni' proposal might serve this role.[1])
You definitely wouldn't use MD5, as experts have been recommending against its use for content-security since about 1996.[2] (A practical full chosen-preimage attack hasn't yet been developed, but still, you'd design for security for the next few decades, which would mean a SHA256 or better.) The choice of a good hash would mean no one could practically create an alternate file with the same hash.
To be precise, while MD5 has been 'broken' in the sense of not meeting its design goals for a long time, and there are now a number of scenarios where attackers can create pairs of files with the same MD5, it is not yet practically possible to create a collision for any arbitrary file (such as jquery.js) on demand. That would be the total 'preimage vulnerability' as mentioned at:
MD5 should absolutely not be used for this content-identification purpose or any other new code... and wise designers haven't been using it for 10+ years. I'm just mentioning this to be precise about the current state of its proven weaknesses.
That's actually a very good idea... Browsers could load the jquery file in your cache by its hash, rather than its URL. No more having 100 copies of jquery.min.js in your cache just because they're from different URLs.
It might have privacy issues, though. Say you copy the HN logo to your server, and then serve it with an hash. You can then tell if the person has visited HN by seeing if their browser asks for the logo.
To solve this problem in the context of HTTP proxying you'd need more than just a way to refer to arbitrary content (which, honestly, doesn't need to be a crypto hash, it just needs to be an arbitrary identifier unique to the user and session). First, consider why we use HTTPS (in an overly-simplistic view):
1. Integrity
You want to make sure the data you're getting is from the source you expect, and that it hasn't been compromised. HTTPS does this with PKI by enabling you to verify the destination host is really who they say they are (Certificates) and to only trade data with them. Anybody who doesn't pass the signed-certificate-verifying test, doesn't get to give us data.
2. Secrecy
We don't want anyone else knowing what our data is because it may contain sensitive information. Once we verify the identity of the sender, each session is independently encrypted to prevent later decoding.
So what would we need to cache our content and retain its integrity and secrecy? The simplest thing would be encrypted blobs of data signed by our destination host's certificate. A proxy could keep data for a set amount of time, perhaps each piece of data encapsulated in a different session. All our client would need to do was connect once and initiate a session, and the server could deliver a copy of the encrypted/signed payload to the proxy.
With some magic flags in the new protocol our client could be instructed that the server allows the client to make a 'proxy request' to the destination for content. This request could be made in such a way that it allows a proxy to intercept this request from the client (which could be plaintext actually), get the encrypted chunk from the destination (which could also be done plaintext), and the proxy could deliver the chunk to the client, similar to what it does now with HTTP. Since the chunk was signed and encrypted by the destination, the proxy can't do anything but deliver the exact copy the destination gave it. Our client receives the data it wants from the proxy and verifies it's from the destination, unpacks it and loads it.
1. Client requests content from server (HTTPS)
2. Server replies back that server allows proxy requests (HTTPS)
3. Client sends request again with proxy-request flags and arbitrary content identifier & session identifier (HTTP)
4. Proxy receives request, gets content from server (HTTP)
5. Proxy replies to client delivering content from server (HTTP)
6. Client verifies content was signed by server
Of course this would be limited in its usefulness compared to plaintext caching; it would be user and session specific, so only lots of requests by the same client in a session would benefit from this. But it would theoretically save on bulk requests of encrypted content while preserving integrity and secrecy.
Comments
Crazy. If you want to use caching, just use HTTP for that content.
It's not that simple.
If you are going to use HTTPS, you need to use it for all content on that domain. Otherwise if you load for example a large javascript file over HTTP, the attacker can just poison that file and control your whole page.
Even if you loaded an image from the same domain, your credentials would sent sent as a cookie in plain text.
You could use a separate domain for content as explained here: http://stackoverflow.com/a/5160657/804713
The web is long overdue for a method to specify an exact resource, by content-hash, from one-of-whatever-sources.
Those sources can then be other less-secure protocols, even those unanticipated by the referrer, because the client got the necessary verifier via the secure-path.
I believe there's already a standard HTTP header for this: Content-MD5.
Browsers definitely don't cache across origins by this though.
If they did, would it be possible to create a malicious JavaScript file with the same MD5 as jQuery?
It would need to be done via the pointer-to-content (URL/URI), and an independently-calculated secure-hash, not just a header. (The 'ni' proposal might serve this role.[1])
You definitely wouldn't use MD5, as experts have been recommending against its use for content-security since about 1996.[2] (A practical full chosen-preimage attack hasn't yet been developed, but still, you'd design for security for the next few decades, which would mean a SHA256 or better.) The choice of a good hash would mean no one could practically create an alternate file with the same hash.
[1] http://tools.ietf.org/html/draft-farrell-decade-ni-10
[2] http://en.wikipedia.org/wiki/MD5#cite_note-30
Yes it would be possible because MD5 has been broken: https://en.wikipedia.org/wiki/MD5#Collision_vulnerabilities
Any file can be modified to result in a hash collision with a specific MD5. This makes is unsuitable for its stated purpose as a cryptographic hash.
The solution would be to use a newer and stronger hash like Keccak.
A solution similar to what you are thinking of is already used by Bittorrent's Distributed Hash Table to identify files.
To be precise, while MD5 has been 'broken' in the sense of not meeting its design goals for a long time, and there are now a number of scenarios where attackers can create pairs of files with the same MD5, it is not yet practically possible to create a collision for any arbitrary file (such as jquery.js) on demand. That would be the total 'preimage vulnerability' as mentioned at:
https://en.wikipedia.org/wiki/MD5#Preimage_vulnerability
MD5 should absolutely not be used for this content-identification purpose or any other new code... and wise designers haven't been using it for 10+ years. I'm just mentioning this to be precise about the current state of its proven weaknesses.
That's actually a very good idea... Browsers could load the jquery file in your cache by its hash, rather than its URL. No more having 100 copies of jquery.min.js in your cache just because they're from different URLs.
Precisely this is actively being discussed in the W3C WebAppSec WG: http://w3c.github.io/webappsec/specs/subresourceintegrity/
The security folks I talk to are... nervous... about this use of subresource integrity, however.
Can you say any more about what makes them nervous? (What do they think will go wrong?)
Early to say... it's their job to be nervous about new things.
It might have privacy issues, though. Say you copy the HN logo to your server, and then serve it with an hash. You can then tell if the person has visited HN by seeing if their browser asks for the logo.
To solve this problem in the context of HTTP proxying you'd need more than just a way to refer to arbitrary content (which, honestly, doesn't need to be a crypto hash, it just needs to be an arbitrary identifier unique to the user and session). First, consider why we use HTTPS (in an overly-simplistic view):
1. Integrity
You want to make sure the data you're getting is from the source you expect, and that it hasn't been compromised. HTTPS does this with PKI by enabling you to verify the destination host is really who they say they are (Certificates) and to only trade data with them. Anybody who doesn't pass the signed-certificate-verifying test, doesn't get to give us data.
2. Secrecy
We don't want anyone else knowing what our data is because it may contain sensitive information. Once we verify the identity of the sender, each session is independently encrypted to prevent later decoding.
So what would we need to cache our content and retain its integrity and secrecy? The simplest thing would be encrypted blobs of data signed by our destination host's certificate. A proxy could keep data for a set amount of time, perhaps each piece of data encapsulated in a different session. All our client would need to do was connect once and initiate a session, and the server could deliver a copy of the encrypted/signed payload to the proxy.
With some magic flags in the new protocol our client could be instructed that the server allows the client to make a 'proxy request' to the destination for content. This request could be made in such a way that it allows a proxy to intercept this request from the client (which could be plaintext actually), get the encrypted chunk from the destination (which could also be done plaintext), and the proxy could deliver the chunk to the client, similar to what it does now with HTTP. Since the chunk was signed and encrypted by the destination, the proxy can't do anything but deliver the exact copy the destination gave it. Our client receives the data it wants from the proxy and verifies it's from the destination, unpacks it and loads it.
Of course this would be limited in its usefulness compared to plaintext caching; it would be user and session specific, so only lots of requests by the same client in a session would benefit from this. But it would theoretically save on bulk requests of encrypted content while preserving integrity and secrecy.credential-containing cookies should be set as secure.
If HTTP 2.0 requires TLS then to get speed and caching you'll need some kind of trusted proxy.