While it is good that Apple seeks formal security analysis over its proposed system, there are some fundamental security assumptions baked in the system:
- It assumes that the server would not tamper its dataset (i,e., the list of CSAM). So it is OK to disclose the information if the client has enough matchings. But in reality, nothing prevents a malicious server adding arbitrary content to the list.
- It fails to consider the vulnerabilities of the perceptual hash. This includes false positives and adversarial collision attacks (https://arxiv.org/abs/2011.09473).
Another potential long term issue is that it is unclear how long Apple will store the safety vouchers. As a storage service, Apple may store them forever. The system is based on Elliptic-curve cryptography. Despite it is the current state-of-the-art encryption technique, it will be broken when the quantum computer becomes a reality in the future. So it is possible that every encrypted safety vouchers can be decrypted in the next 50 years.
Some apps like WhatsApp save images from messages automatically to the camera roll.
So a theoretical attacker could send you a message on WA with a set of adversarial collision images (not even child pornography potentially), and effectively SWAT you
Comments
While it is good that Apple seeks formal security analysis over its proposed system, there are some fundamental security assumptions baked in the system:
- It assumes that the server would not tamper its dataset (i,e., the list of CSAM). So it is OK to disclose the information if the client has enough matchings. But in reality, nothing prevents a malicious server adding arbitrary content to the list.
- It fails to consider the vulnerabilities of the perceptual hash. This includes false positives and adversarial collision attacks (https://arxiv.org/abs/2011.09473).
Another potential long term issue is that it is unclear how long Apple will store the safety vouchers. As a storage service, Apple may store them forever. The system is based on Elliptic-curve cryptography. Despite it is the current state-of-the-art encryption technique, it will be broken when the quantum computer becomes a reality in the future. So it is possible that every encrypted safety vouchers can be decrypted in the next 50 years.
Some apps like WhatsApp save images from messages automatically to the camera roll.
So a theoretical attacker could send you a message on WA with a set of adversarial collision images (not even child pornography potentially), and effectively SWAT you
This is already possible on both iOS (iCloud Photos) and Android (Google Photos) as both scan for CSAM server side.
Google photos does not by default upload photos in WhatsApp folder till you enable it, default is no, so no...
And everyone will be on the list as pedophiles.