Filtering to target at high-risk sub-groups - which, among themselves, have different base rates than the general population- is entirely effective.
It is precisely the reasoning under which medical screening - ineffective, generally, due to the base rate - is aimed at, say, "Men Over 50", where the base rate grows sufficiently large to make screening worthwhile.
What trouble do you have with the fact that there aren't 100 terrorist plots in NYC? There might not even be 2 of them. If they're filtering down to 100 visits a week, something close to 99% of the visits are false positives.
tptacek seems hell-bent on not giving the side for which he has no factual data any mindshare at all--there are arguments for that mindset.
As for your filtering observation, yeah, I have no idea if he just mis-stated his position, is using different definitions, or today just can't into math.
One of two possibilities arise from this comment. The first is that you think that there might be many many terrorist plots ongoing at any time. The other is that you don't understand the simple idea of a base rate and how it applies to statistical filters.
It's unfortunate, because that article and the base rate article have some contradictory conclusions--but, the linked article shows its math.
What's really troubling me here though is that you seem unwilling even as a thought experiment to entertain the notion that this sort of behavior could be purposeful by the government. In many, many prior discussions you've held this view, you've advocated for stripping rights, you've advocated for screwing over people scraping public unauthenticated APIs, you've advocated a great many things against the general notion of the public good, of privacy, and of exploration.
Perhaps come down here and--in the name of intellectual honesty--try and understand what exactly has so many of these people bothered. As it is, due to your choice of message and presentation, you often come off as at best aloof and at worst willfully ignorant.
I like how this comment goes from acknowledging that the author didn't know about an extremely basic and fundamental concept of statistics to a whole paragraph of (frankly weird) assertions about my character and beliefs. It's almost gymnastic.
I think one place to start moving forward is to acknowledge that you really have no idea what I do or don't believe.
Comments
I do not think so.
Filtering to target at high-risk sub-groups - which, among themselves, have different base rates than the general population- is entirely effective.
It is precisely the reasoning under which medical screening - ineffective, generally, due to the base rate - is aimed at, say, "Men Over 50", where the base rate grows sufficiently large to make screening worthwhile.
What trouble do you have with the fact that there aren't 100 terrorist plots in NYC? There might not even be 2 of them. If they're filtering down to 100 visits a week, something close to 99% of the visits are false positives.
tptacek seems hell-bent on not giving the side for which he has no factual data any mindshare at all--there are arguments for that mindset.
As for your filtering observation, yeah, I have no idea if he just mis-stated his position, is using different definitions, or today just can't into math.
One of two possibilities arise from this comment. The first is that you think that there might be many many terrorist plots ongoing at any time. The other is that you don't understand the simple idea of a base rate and how it applies to statistical filters.
So, your first possibility is incorrect.
The second is worth investigating, and in the interest of teaching others I'll provide a helpful link:
http://en.wikipedia.org/wiki/Base_rate_fallacy
It's unfortunate, because that article and the base rate article have some contradictory conclusions--but, the linked article shows its math.
What's really troubling me here though is that you seem unwilling even as a thought experiment to entertain the notion that this sort of behavior could be purposeful by the government. In many, many prior discussions you've held this view, you've advocated for stripping rights, you've advocated for screwing over people scraping public unauthenticated APIs, you've advocated a great many things against the general notion of the public good, of privacy, and of exploration.
Perhaps come down here and--in the name of intellectual honesty--try and understand what exactly has so many of these people bothered. As it is, due to your choice of message and presentation, you often come off as at best aloof and at worst willfully ignorant.
I like how this comment goes from acknowledging that the author didn't know about an extremely basic and fundamental concept of statistics to a whole paragraph of (frankly weird) assertions about my character and beliefs. It's almost gymnastic.
I think one place to start moving forward is to acknowledge that you really have no idea what I do or don't believe.