I am not so sure why you think this is bizarre. In an ideal world TCP is reliable, HTTP assumes it runs on reliable transport so it doesn't do any more integrity checking.
In the real world there are bugs in TCP stacks and in HTTP implementations that cause HTTP traffic to get corrupted. I see this every day. Some applications do implement extra checking, most applications do not. Browsers and wget and curl can't implement any extra checking because the way you check is application specific. There is no standard way to do it; what you mention there is an esoteric feature.
Just for anecdotal fun, Logic Pro tries to download over 50GB of assets over HTTP (each individual file is many-GB). It has never worked for me over any of my networks (and in fact I wrote a tool to fix this).
In the real world there are bugs in TCP stacks and in HTTP implementations that cause HTTP traffic to get corrupted
I'm very much aware of this. Hence why I expressed surprise that such checksumming was not commonly performed at the application layer.
There is no standard way to do it; what you mention there is an esoteric feature.
The Content-MD5 header is defined in RFC 2616. It is, by definition, standard. If it's not widely supported, then I think that it would behoove the people who care about these things to switch to servers/clients which do support it.
(I suspect the intersection between "people who know/care how to use md5sum" and "people who know how to set/read an arbitrary HTTP header" is fairly large. Hence my surprise at the common practice of ignoring this capability.)
Comments
I am not so sure why you think this is bizarre. In an ideal world TCP is reliable, HTTP assumes it runs on reliable transport so it doesn't do any more integrity checking.
In the real world there are bugs in TCP stacks and in HTTP implementations that cause HTTP traffic to get corrupted. I see this every day. Some applications do implement extra checking, most applications do not. Browsers and wget and curl can't implement any extra checking because the way you check is application specific. There is no standard way to do it; what you mention there is an esoteric feature.
Just for anecdotal fun, Logic Pro tries to download over 50GB of assets over HTTP (each individual file is many-GB). It has never worked for me over any of my networks (and in fact I wrote a tool to fix this).
I'm very much aware of this. Hence why I expressed surprise that such checksumming was not commonly performed at the application layer.
The Content-MD5 header is defined in RFC 2616. It is, by definition, standard. If it's not widely supported, then I think that it would behoove the people who care about these things to switch to servers/clients which do support it.
(I suspect the intersection between "people who know/care how to use md5sum" and "people who know how to set/read an arbitrary HTTP header" is fairly large. Hence my surprise at the common practice of ignoring this capability.)