Yes; if you increase the hash space sufficiently, these problems go away. I don't think wide hashing has really become a standard industry practice though, because services want to pick the option that is least burdensome to end users' CPUs. Another issue is that once they have a big de-duplication database established based on a particular hash, switching is expensive. I suspect a lot of shops are using md5 still.
Perhaps. That said, even a SHA-512 sum on my computer seems to take a little less than a CPU-second per hundred megabytes. Odds are you're not going to be uploading that fast, so you should be able to do that work "online" and not have a noticeable impact on either upload throughput or user-visible impact. This is doubly true on a multi-core device, since the sha sum I quoted was single-threaded. I would think the more important thing to minimize is user-impacting disk latency from the backup scan.
Comments
Yes; if you increase the hash space sufficiently, these problems go away. I don't think wide hashing has really become a standard industry practice though, because services want to pick the option that is least burdensome to end users' CPUs. Another issue is that once they have a big de-duplication database established based on a particular hash, switching is expensive. I suspect a lot of shops are using md5 still.
Perhaps. That said, even a SHA-512 sum on my computer seems to take a little less than a CPU-second per hundred megabytes. Odds are you're not going to be uploading that fast, so you should be able to do that work "online" and not have a noticeable impact on either upload throughput or user-visible impact. This is doubly true on a multi-core device, since the sha sum I quoted was single-threaded. I would think the more important thing to minimize is user-impacting disk latency from the backup scan.