Privacy for the server: A malicious client should learn nothing about the server’s dataset X ⊆ U other than its size. In particular, it is important that the client learn nothing about the intersection size |id(Y̅ ∩ X)|. Otherwise, the client can use that to extract information about X by adding test items to its list Y̅, and checking if the intersection size changes.
If one of your pictures is a false positive hash collision, you'll have no idea until your front door gets broken down.
Privacy for the client: Let X be the server’s input from which pdata is derived. A malicious server must learn nothing about the client’s Y̅ beyond the output of ftPSIAD with respect to this set X.
Apple can't check whether a hash match is a false positive or not, because they only get the matching hashes and not the pictures that triggered them. So if you have a bunch of false positives, your front door is getting broken down, with no opportunity for a human to realize the problem and intervene.
The protocol need not provide correct output against a malicious client. That is, the protocol need not prevent a malicious client from causing the server to obtain an incorrect ftPSI-AD output when the protocol terminates. The reason for this is that a malicious client can always choose to hide some of its data from the PSI system in order to cause an undercount of the intersection.
Their protocol isn't (and can't be) secure against the one attack that the people that this system is supposed to catch would actually commit.
Moreover, a malicious client that attempts to cause an overcount of the intersection will be detected by mechanisms outside of the cryptographic protocol.
This seems eerie to me but I can't put my finger on why.
"So if you have a bunch of false positives, your front door is getting broken down, with no opportunity for a human to realize the problem and intervene."
Well, technically speaking, once enough security vouchers have been submitted and reached the threshold as stated, a report will be sent to an Apple employee (somewhere). The vouchers will then, combined, be decryptable and contain grayscale low-res versions of the original image for confirmation, in which case the NCEMC (National Center for Exploited and Missing Children) will be alerted and law enforcement.
I'm just pointing out that your door being broke down is by multiple false flags and a human will get a chance to "realize the problem and intervene" before it goes to the FBI or whatever. Not saying I like this system, just making a nitpick on your criticism. I don't really know how else you could have a human intervene without "breaking down your door."
...be decryptable and contain grayscale low-res versions of the original image for confirmation...
But what happens when the false positives are erroneously confirmed as legitimate CSAM? What's the system in place for removing all the security vouchers on someone's account because the vision system flagged a bunch of false positives? What's the process for unfucking a person's life because the employee confirming the CSAM was in a bad mood that day? Is Apple going to pay the legal bills of someone they effectively SWATed?
For the ones of people this system might actually catch with legitimate CSAM there will be at least as many false positives slipping through to ruinous consequences. Law enforcement shouldn't be trusted as a backstop against abuse because LEOs and DAs are incentivized for "good numbers" and "results", not for actually meting out justice. If someone flagged by false positives gets to the stage of law enforcement being involved their lives will be ruined.
"For the ones of people this system might actually catch with legitimate CSAM there will be at least as many false positives slipping through to ruinous consequences."
Apple claims without math that it's a 1-in-a-trillion chance that your account will be incorrectly flagged.
And even then... what happens? Does the FBI actually smash down your door with a SWAT team for CSAM? In most cases, No. They go and arrest you the normal way. Unless you had a ludicrous amount of false flags (in the thousands) showing you were a dealer and had a previous history.
If one of your pictures is a false positive hash collision, you'll have no idea until your front door gets broken down.
This is a completely false statement. Besides the fact that it requires more than one match for an account to be flagged and flagged accounts are reviewed manually by Apple before sending a report off to NCMEC thereby catching false positives, you’re notified if your account is flagged and can file a challenge.
Comments
If one of your pictures is a false positive hash collision, you'll have no idea until your front door gets broken down.
Apple can't check whether a hash match is a false positive or not, because they only get the matching hashes and not the pictures that triggered them. So if you have a bunch of false positives, your front door is getting broken down, with no opportunity for a human to realize the problem and intervene.
Their protocol isn't (and can't be) secure against the one attack that the people that this system is supposed to catch would actually commit.
This seems eerie to me but I can't put my finger on why.
"So if you have a bunch of false positives, your front door is getting broken down, with no opportunity for a human to realize the problem and intervene."
Well, technically speaking, once enough security vouchers have been submitted and reached the threshold as stated, a report will be sent to an Apple employee (somewhere). The vouchers will then, combined, be decryptable and contain grayscale low-res versions of the original image for confirmation, in which case the NCEMC (National Center for Exploited and Missing Children) will be alerted and law enforcement.
I'm just pointing out that your door being broke down is by multiple false flags and a human will get a chance to "realize the problem and intervene" before it goes to the FBI or whatever. Not saying I like this system, just making a nitpick on your criticism. I don't really know how else you could have a human intervene without "breaking down your door."
But what happens when the false positives are erroneously confirmed as legitimate CSAM? What's the system in place for removing all the security vouchers on someone's account because the vision system flagged a bunch of false positives? What's the process for unfucking a person's life because the employee confirming the CSAM was in a bad mood that day? Is Apple going to pay the legal bills of someone they effectively SWATed?
For the ones of people this system might actually catch with legitimate CSAM there will be at least as many false positives slipping through to ruinous consequences. Law enforcement shouldn't be trusted as a backstop against abuse because LEOs and DAs are incentivized for "good numbers" and "results", not for actually meting out justice. If someone flagged by false positives gets to the stage of law enforcement being involved their lives will be ruined.
How are you calculating this?
Apple claims without math that it's a 1-in-a-trillion chance that your account will be incorrectly flagged.
And even then... what happens? Does the FBI actually smash down your door with a SWAT team for CSAM? In most cases, No. They go and arrest you the normal way. Unless you had a ludicrous amount of false flags (in the thousands) showing you were a dealer and had a previous history.
An Apple employee and a district judge who has been convinced there is probable cause.
I guess they have to call them "employees" in California now. Do they do this before or after becoming Facebook moderators?
This is a completely false statement. Besides the fact that it requires more than one match for an account to be flagged and flagged accounts are reviewed manually by Apple before sending a report off to NCMEC thereby catching false positives, you’re notified if your account is flagged and can file a challenge.