I really feel like SnapChat is fumbling this whole thing. They ignored the security warning, and now seem to be blaming the security group for the leak of info:
On Christmas Eve, that same group publicly documented our API, making it easier for individuals to abuse our service and violate our Terms of Use.
The funny thing is that folks on HN and in the tech community generally will fault SnapChat for their callous attitude to security and pitiful response. But 99.9% of their users won't know or care, and investors will consider this a "lesson learned" and move on without a second thought.
Once the 24/hr news-cycle moves past the hyperbolic "SnapChat Hacked!" headlines, this ordeal and their pathetic response will slip into the forgotten-ether of low-impact data leaks.
How would you have designed this functionality in a way that isn't vulnerable to the same attack?
Rate-limiting can slow you down, but you could run the script for months if you wanted. Fundamentally Snapchat is using user-supplied data, and users can lie.
Maybe we can check if they are lying by querying a database of people who have verified that they know each other? Oh wait, that's Facebook. HN would be boycotting on principle and screaming about violation of privacy.
You seem to take position to defend the snapchat stance ... I read it as a validation that there is a tradeoff made by design, and anyone caring enough about privacy wouldn't have implemented the feature in the first place.
Privacy doesn't have to be the goal of every service, let's just admit that it not a main concern of Snapchat either.
Of course Facebook also has a lot of leaky behaviors concerning finding friends and friend suggestion. They also value service growth more than privacy, and that's their right.
Put hard limits on the rate or total number of friend lookups per account, especially if a lookup fails. Keep track of global lookups and lock down the global lookup rate if there is a large number of failures. Keep track of patterns in lookups. Disallow more than a few lookups for accounts that haven't sent or received any messages. Require approval by the receiver for a lookup based on a phone number before the username is revealed. Charge money for phone number based lookups or require some sort of identification. Only allow phone number based lookups from within the smartphone app, not via the API.
And so on.
This is not rocket science, it requires mainly only that the devs legitimately care about security and privacy. But if the problem is instead approached by cavalier developers who are only reluctantly tackling the problem after having been burned by bad PR due to previous breaches then the result will often be half-assed engineering that does little to solve the problem.
The biggest issue for SnapChat is, to be frank, they are a bunch of dumb-asses who got lucky and managed to cobble together a silly app that a lot of people wanted to use. However, the foundation their app is built on is a degree of privacy, but that privacy is just an illusion, it is not based on any substantive engineering whatsoever and the SnapChat developers are too clueless to actually take on the problems that need tackling. This is just the first of what will ultimately be many failures at SnapChat which will ultimately reveal the trust placed in the company to be completely misplaced, causing their user base to evaporate. They should have taken the acquisition offer, at this rate the chances of their company being in ruins in a few months time is very, very high.
Put hard limits on the rate or total number of friend lookups per account, especially if a lookup fails.
Easy: distribute the search across accounts. Also you'd expect most of the lookups to fail (most people in your contacts won't be Snapchat users).
Keep track of global lookups and lock down the global lookup rate if there is a large number of failure
Again, you expect a large number of failures.
Disallow more than a few lookups for accounts that haven't sent or received any messages.
Using the find friends functionality is intended to get you started with Snapchat. When you haven't sent many/any messages is when it matters most.
Require approval by the receiver for a lookup based on a phone number before the username is revealed.
Many users (myself included) would consider notifications of this volume to be spam.
Charge money for phone number based lookups
I don't know anyone who would pay to use Snapchat.
require some sort of identification
Yeah, because requiring strong identification totally makes privacy advocates happy.
Only allow phone number based lookups from within the smartphone app, not via the API.
Smartphone app has to communicate with Snapchat somehow, so the functionality is going to have to be exposed via an API. iOS is closed enough that you might be able to securely authenticate the app to the API, but AFAIK you can't stop the user from getting the key out of an Android app if he has root.
I can't comment on Snapchat's competence overall, but none of your solutions actually work. This is a nontrivial problem.
>The biggest issue for SnapChat is, to be frank, they are a bunch of dumb-asses who got lucky and managed to cobble together a silly app that a lot of people wanted to use. However, the foundation their app is built on is a degree of privacy, but that privacy is just an illusion, it is not based on any substantive engineering whatsoever and the SnapChat developers are too clueless to actually take on the problems that need tackling.
It depends on what kind of privacy you care about. Personally, I care very much about the privacy of my private communications (Facebook messages, emails, texts).
I don't consider usernames or phone numbers to be private information. Being able to find someone's contact info by name (and vice versa) is useful and people who make this difficult are just annoying. Failing to protect contact information is very different from failing to protect actual communications or metadata, and a callous attitude towards the security of contact information does not imply a callous attitude towards the security of communications.
There is a reasonable argument that a map between phone numbers and usernames could blow pseudonyms. But AFAIK most people use Snapchat with their friends - people who already have their names and phone numbers. If you reuse your Snapchat username somewhere where you want to be anonymous, you've already fucked up. Your cover is already blown. Whether it's blown to your friends or the whole world is a distinction without a difference.
AFAIK, it falls under PII. I realize some may not consider this personal info, but many others do. There are laws built around its protection and disclosure.
I think a failure to protect any data, including PII, is harmful to the user experience and a brand's image.
Rate-limiting can slow you down, but you could run the script for months if you wanted. Fundamentally Snapchat is using user-supplied data, and users can lie.
You can make the same argument about employing exponential backoff on a login screen. It's still very effective deterrence.
There is a good chance that a failed password attempt is an indication of something shady going on, so exponential delays are appropriate.
How are you going to tell the difference between an API hit for the phone number of someone you know vs. a phone number you made up? Or are you going to increase the delay after every query? If you do that, then the "find friends" feature will just break down for people with sufficiently large contact lists.
Maybe don't reveal the username unless that user has your phone number in their contacts? Sounds good. But how are you going to check? Snapchat would need to store every user's contact list, which HN also considers unacceptable. Hashing is not an effective mitigator here because it's so easy to bruteforce the space of 10-digit numbers.
Maybe rate-limiting as such is not the right idea, but how about just limiting accounts? I think there is a substantial difference between the 4.6mm collected and even a "sufficiently large contact list." There must be a number at which you can be banned from requesting any more.
Maybe don't reveal the username unless that user has your phone number in their contacts? Sounds good. But how are you going to check? Snapchat would need to store every user's contact list, which HN also considers unacceptable. Hashing is not an effective mitigator here because it's so easy to bruteforce the space of 10-digit numbers.
There's a partial fix to this that minimises what is stored by Snapchat. It does introduce a delay as it requires the other user to be logged in or eventually log in:-
User_A joins and has 10 numbers in his contacts list and Snapchat wants to find if they are friends (i.e. they both have each other's phone number in their contact lists). All 10 numbers are provided to Snapchat, only 5 numbers correspond to existing Snapchat users, the other 5 numbers are discarded by Snapchat. These remaining numbers, now linked to snapchat Usernames, then go into a table:-
When User_B next logs in the app checks for any people seeking connection with them; an API call returns a list of phone numbers which the app should check (i.e. User_B executing this would get the phone number of User_A) to see if they are in the local contact list (rather than uploading their entire contact list to Snapchat). If any numbers are in that list then they can be provided back to Snapchat and the server examines the table to find out which friends can be paired together, upon return of these results all rows for people seeking User_B will be either actioned (in User_B's contact list) or cleared (not in the contact list).
This means only a subset of a user's contact list is stored (the numbers that are known to use Snapchat), and only until the other party logs in, all of which is akin to an automated 'friend request'.
No persistent storage of all users' contact lists required, nor is there any need to store anything else about the contact list other than it contained the phone number of an existing Snapchat user.
Entries in this DB would timeout (pick an appropriate timeout) so they are not retained in perpetuity, new entries would only be created if the other user (i.e. User_B) has been active in Snapchat recently.
The app would also monitor for additions/modifications to the contact list on the phone and attempt new friend discovery by the same method. (This can be done via hashing, for each contact the app stores locally just a hash of the contact's information - name and phone numbers at least - and then on each startup (or at a certain interval) it compares the hashes it knows about with the hashes obtained from the current contact list, the friend discovery is then performed for any contacts relating to new hashes.)
User_A doesn't need to worry about missing users who join subsequently (i.e. one of the 5 numbers that were unknown to Snapchat when they first joined) as these will be handled by the subsequent user joining and going through this same process.
There aren't great simple solutions to this issue AFAIK, but here's my take since I had a similar android app feature:
Want to find friends using contacts in phone:
Snapchat solution: pass number(s), return user info if number(s) in DB. +rate limit
Problem: rate limiting is not enough in this case.
Other solution:
Save contact lists of two users. If user requests friend look up, compare contact lists.
Basically return user info if the two numbers have each other in contact list, thus a verification they know each other.
Duping the snapchat system with the old method won't work, but drawbacks are that you have to store
a copy of user sensitive information, and if that gets out from your servers in some other hack, that would be
really bad, as bad if not worse than the first since it has more information, i.e. who knows who. Also less efficient,
space and time wise.
HN maintains that this is unethical because the users in your contact list have not consented to having their phone numbers shared with a social networking service.
You could use a key derivation function (scrypt or pbkdf2) to expand it out and make it quite difficult to even rainbow table it. You could also compound the effort by only storing a hash of the pairs (one hash or derived key of two phone numbers) with no associated metadata directly linked to it, only as an access gate. Collisions may occur but at least the positive results from a total search space would be significantly reduced.
I Am Not A Cryptographer, so I admit there is likely faulty reasoning in there somewhere, though.
The point is that the hash is effectively reversible at that scale. 10 numeric digits is 10 billion possibilities. It's actually far less than that because area codes don't span the entire 3-digit number range, and most area codes are not heavily populated. But even so, creating a lookup table by computing 10 billion hashes is today a comparatively easy task, even for expensive hash algorithms. It's a matter of only a few seconds to minutes work with relatively common hardware (high performance GPUs).
If only it would go away so easily. The PR strategy they have going on, if taken at face value, is a sure way to have stuff like this happen again; and every time they'll be the losers. Pissing off the same people who are trying to help you is childish at best.
Exactly. Getting individual users to care about privacy or security is ultimately a cultural thing. In the US (which probably has most of Snapchat's users), the attitude of the general public to this seems to be pretty gung-ho, leading to this being a non-event.
I suspect (but can't be sure) that if something like this happened, in, say, Germany, it would generate far more impact and press attention. Most of my US acquaintances who are college students don't give two hoots about privacy -- they are far too busy marketing themselves to the world as much as possible.
Comments
I really feel like SnapChat is fumbling this whole thing. They ignored the security warning, and now seem to be blaming the security group for the leak of info:
The funny thing is that folks on HN and in the tech community generally will fault SnapChat for their callous attitude to security and pitiful response. But 99.9% of their users won't know or care, and investors will consider this a "lesson learned" and move on without a second thought.
Once the 24/hr news-cycle moves past the hyperbolic "SnapChat Hacked!" headlines, this ordeal and their pathetic response will slip into the forgotten-ether of low-impact data leaks.
How would you have designed this functionality in a way that isn't vulnerable to the same attack?
Rate-limiting can slow you down, but you could run the script for months if you wanted. Fundamentally Snapchat is using user-supplied data, and users can lie.
Maybe we can check if they are lying by querying a database of people who have verified that they know each other? Oh wait, that's Facebook. HN would be boycotting on principle and screaming about violation of privacy.
You seem to take position to defend the snapchat stance ... I read it as a validation that there is a tradeoff made by design, and anyone caring enough about privacy wouldn't have implemented the feature in the first place.
Privacy doesn't have to be the goal of every service, let's just admit that it not a main concern of Snapchat either.
Of course Facebook also has a lot of leaky behaviors concerning finding friends and friend suggestion. They also value service growth more than privacy, and that's their right.
Put hard limits on the rate or total number of friend lookups per account, especially if a lookup fails. Keep track of global lookups and lock down the global lookup rate if there is a large number of failures. Keep track of patterns in lookups. Disallow more than a few lookups for accounts that haven't sent or received any messages. Require approval by the receiver for a lookup based on a phone number before the username is revealed. Charge money for phone number based lookups or require some sort of identification. Only allow phone number based lookups from within the smartphone app, not via the API.
And so on.
This is not rocket science, it requires mainly only that the devs legitimately care about security and privacy. But if the problem is instead approached by cavalier developers who are only reluctantly tackling the problem after having been burned by bad PR due to previous breaches then the result will often be half-assed engineering that does little to solve the problem.
The biggest issue for SnapChat is, to be frank, they are a bunch of dumb-asses who got lucky and managed to cobble together a silly app that a lot of people wanted to use. However, the foundation their app is built on is a degree of privacy, but that privacy is just an illusion, it is not based on any substantive engineering whatsoever and the SnapChat developers are too clueless to actually take on the problems that need tackling. This is just the first of what will ultimately be many failures at SnapChat which will ultimately reveal the trust placed in the company to be completely misplaced, causing their user base to evaporate. They should have taken the acquisition offer, at this rate the chances of their company being in ruins in a few months time is very, very high.
Easy: distribute the search across accounts. Also you'd expect most of the lookups to fail (most people in your contacts won't be Snapchat users).
Again, you expect a large number of failures.
Using the find friends functionality is intended to get you started with Snapchat. When you haven't sent many/any messages is when it matters most.
Many users (myself included) would consider notifications of this volume to be spam.
I don't know anyone who would pay to use Snapchat.
Yeah, because requiring strong identification totally makes privacy advocates happy.
Smartphone app has to communicate with Snapchat somehow, so the functionality is going to have to be exposed via an API. iOS is closed enough that you might be able to securely authenticate the app to the API, but AFAIK you can't stop the user from getting the key out of an Android app if he has root.
I can't comment on Snapchat's competence overall, but none of your solutions actually work. This is a nontrivial problem.
Well said, you nailed it.
>The biggest issue for SnapChat is, to be frank, they are a bunch of dumb-asses who got lucky and managed to cobble together a silly app that a lot of people wanted to use. However, the foundation their app is built on is a degree of privacy, but that privacy is just an illusion, it is not based on any substantive engineering whatsoever and the SnapChat developers are too clueless to actually take on the problems that need tackling.
Correct me if I'm wrong, but I don't think they have a public API. The API the apps use is was reverse-engineered and exploited.
It depends on what kind of privacy you care about. Personally, I care very much about the privacy of my private communications (Facebook messages, emails, texts).
I don't consider usernames or phone numbers to be private information. Being able to find someone's contact info by name (and vice versa) is useful and people who make this difficult are just annoying. Failing to protect contact information is very different from failing to protect actual communications or metadata, and a callous attitude towards the security of contact information does not imply a callous attitude towards the security of communications.
There is a reasonable argument that a map between phone numbers and usernames could blow pseudonyms. But AFAIK most people use Snapchat with their friends - people who already have their names and phone numbers. If you reuse your Snapchat username somewhere where you want to be anonymous, you've already fucked up. Your cover is already blown. Whether it's blown to your friends or the whole world is a distinction without a difference.
AFAIK, it falls under PII. I realize some may not consider this personal info, but many others do. There are laws built around its protection and disclosure.
I think a failure to protect any data, including PII, is harmful to the user experience and a brand's image.
You can make the same argument about employing exponential backoff on a login screen. It's still very effective deterrence.
There is a good chance that a failed password attempt is an indication of something shady going on, so exponential delays are appropriate.
How are you going to tell the difference between an API hit for the phone number of someone you know vs. a phone number you made up? Or are you going to increase the delay after every query? If you do that, then the "find friends" feature will just break down for people with sufficiently large contact lists.
Maybe don't reveal the username unless that user has your phone number in their contacts? Sounds good. But how are you going to check? Snapchat would need to store every user's contact list, which HN also considers unacceptable. Hashing is not an effective mitigator here because it's so easy to bruteforce the space of 10-digit numbers.
Maybe rate-limiting as such is not the right idea, but how about just limiting accounts? I think there is a substantial difference between the 4.6mm collected and even a "sufficiently large contact list." There must be a number at which you can be banned from requesting any more.
There's a partial fix to this that minimises what is stored by Snapchat. It does introduce a delay as it requires the other user to be logged in or eventually log in:-
User_A joins and has 10 numbers in his contacts list and Snapchat wants to find if they are friends (i.e. they both have each other's phone number in their contact lists). All 10 numbers are provided to Snapchat, only 5 numbers correspond to existing Snapchat users, the other 5 numbers are discarded by Snapchat. These remaining numbers, now linked to snapchat Usernames, then go into a table:-
When User_B next logs in the app checks for any people seeking connection with them; an API call returns a list of phone numbers which the app should check (i.e. User_B executing this would get the phone number of User_A) to see if they are in the local contact list (rather than uploading their entire contact list to Snapchat). If any numbers are in that list then they can be provided back to Snapchat and the server examines the table to find out which friends can be paired together, upon return of these results all rows for people seeking User_B will be either actioned (in User_B's contact list) or cleared (not in the contact list).This means only a subset of a user's contact list is stored (the numbers that are known to use Snapchat), and only until the other party logs in, all of which is akin to an automated 'friend request'.
No persistent storage of all users' contact lists required, nor is there any need to store anything else about the contact list other than it contained the phone number of an existing Snapchat user.
Entries in this DB would timeout (pick an appropriate timeout) so they are not retained in perpetuity, new entries would only be created if the other user (i.e. User_B) has been active in Snapchat recently.
The app would also monitor for additions/modifications to the contact list on the phone and attempt new friend discovery by the same method. (This can be done via hashing, for each contact the app stores locally just a hash of the contact's information - name and phone numbers at least - and then on each startup (or at a certain interval) it compares the hashes it knows about with the hashes obtained from the current contact list, the friend discovery is then performed for any contacts relating to new hashes.)
User_A doesn't need to worry about missing users who join subsequently (i.e. one of the 5 numbers that were unknown to Snapchat when they first joined) as these will be handled by the subsequent user joining and going through this same process.
There aren't great simple solutions to this issue AFAIK, but here's my take since I had a similar android app feature:
Want to find friends using contacts in phone:
Snapchat solution: pass number(s), return user info if number(s) in DB. +rate limit
Problem: rate limiting is not enough in this case.
Other solution:
Save contact lists of two users. If user requests friend look up, compare contact lists.
Basically return user info if the two numbers have each other in contact list, thus a verification they know each other.
Duping the snapchat system with the old method won't work, but drawbacks are that you have to store a copy of user sensitive information, and if that gets out from your servers in some other hack, that would be really bad, as bad if not worse than the first since it has more information, i.e. who knows who. Also less efficient, space and time wise.
HN maintains that this is unethical because the users in your contact list have not consented to having their phone numbers shared with a social networking service.
Hash the numbers and compare hashes instead.
10 digits isn't long enough for that to work.
really? there are no hashing algorithms where 10 digits is enough?
10 numeric digits is only 10 billion, so, no, even if the algorithm took many hundreds of milliseconds.
You could use a key derivation function (scrypt or pbkdf2) to expand it out and make it quite difficult to even rainbow table it. You could also compound the effort by only storing a hash of the pairs (one hash or derived key of two phone numbers) with no associated metadata directly linked to it, only as an access gate. Collisions may occur but at least the positive results from a total search space would be significantly reduced.
I Am Not A Cryptographer, so I admit there is likely faulty reasoning in there somewhere, though.
https://whispersystems.org/blog/contact-discovery/
The point is that the hash is effectively reversible at that scale. 10 numeric digits is 10 billion possibilities. It's actually far less than that because area codes don't span the entire 3-digit number range, and most area codes are not heavily populated. But even so, creating a lookup table by computing 10 billion hashes is today a comparatively easy task, even for expensive hash algorithms. It's a matter of only a few seconds to minutes work with relatively common hardware (high performance GPUs).
If only it would go away so easily. The PR strategy they have going on, if taken at face value, is a sure way to have stuff like this happen again; and every time they'll be the losers. Pissing off the same people who are trying to help you is childish at best.
Exactly. Getting individual users to care about privacy or security is ultimately a cultural thing. In the US (which probably has most of Snapchat's users), the attitude of the general public to this seems to be pretty gung-ho, leading to this being a non-event.
I suspect (but can't be sure) that if something like this happened, in, say, Germany, it would generate far more impact and press attention. Most of my US acquaintances who are college students don't give two hoots about privacy -- they are far too busy marketing themselves to the world as much as possible.