No, they normally never control for anything that smells of class (like income) because if they do, the health effect disappears. There was a good recent case or rerunning a study and controlling for class, thereby eliminating an effect (I think it was about potatoes and weight gain or something.) Don't have access to it here, but I'll try to post it when I get home.
IMO 95% of the health studies that catch fire in the popular press are repeated rediscoveries of class (smoking, fast food and types of food, etc.), or the qualities of illness (people who sit more die earlier, people who are depressed die earlier, etc.)
They did mention controlling for smoking, though, because they found that it counted against them.
Thanks for that. It's a little annoying that every time a study like this comes up, somebody jumps in with "Well, did they control for [obvious thing to control for]?"
You don't become a scientist because you're lazy and stupid. If you're reading about a study and think there's an obvious reason it's inaccurate, you are almost certainly wrong. Not that the study is not inaccurate -- many, perhaps most are -- but the reason is definitely not obvious. It's something that the authors, people who have spent a lot more time thinking about it than you, and who are probably smarter and better educated to boot, never figured out.
Guess what: it looks like the authors didn't control for income (or even just whether the person has a job!) or intelligence.
I just read the paper; there are many diet and exercise-related controls, and that's it as far as I can see. I totally agree with the OP that being the kind of person who might drink coffee probably has a confounding effect on health. I can't even think of how to try to control it properly.
I don't understand how you get from "most studies are inaccurate" to "but you shouldn't assume the ones that get reported on in the media are inaccurate in a way that we can detect". Why not? People who are performing studies that get picked up by popular media are the least likely people to be performing their work rigorously, and the most likely to be choosing a sensational hypothesis that's difficult to measure correctly.
I agree that we might eventually get to a point where the amount of skepticism and cynicism about scientific results vastly outweighs the number of papers published with poor experimental design or lack of real statistical significance. But I don't think we're currently anywhere close!
You make some good points, so I hope this doesn't seem dismissive-- my claim is not that the study is right. My claim is simply that the editors of the NEJM decided to publish the study having read it in greater depth than any of us here would care to, so if I read the CNN article and think there's an obvious reason it's not true, I can be pretty sure I'm wrong.
Now that being said, I think we're talking at cross purposes. The study found a link between coffee drinknig and mortality; I misread the GGP as being critical of that link, where it's actually just a hypothesis which would explain it. So none of this is really relevant :P
I read the "methods" section of the study, and saw no reference to controlling for occupation nor socioeconomic status (as opposed to marital status, self-reported health, and other things that are listed).
However, I only read it briefly, so I may have missed something.
Comments
Like almost everyone else who will comment here, I haven't read the study, but studies normally statistically control for these sorts of things.
No, they normally never control for anything that smells of class (like income) because if they do, the health effect disappears. There was a good recent case or rerunning a study and controlling for class, thereby eliminating an effect (I think it was about potatoes and weight gain or something.) Don't have access to it here, but I'll try to post it when I get home.
IMO 95% of the health studies that catch fire in the popular press are repeated rediscoveries of class (smoking, fast food and types of food, etc.), or the qualities of illness (people who sit more die earlier, people who are depressed die earlier, etc.)
They did mention controlling for smoking, though, because they found that it counted against them.
Thanks for that. It's a little annoying that every time a study like this comes up, somebody jumps in with "Well, did they control for [obvious thing to control for]?"
You don't become a scientist because you're lazy and stupid. If you're reading about a study and think there's an obvious reason it's inaccurate, you are almost certainly wrong. Not that the study is not inaccurate -- many, perhaps most are -- but the reason is definitely not obvious. It's something that the authors, people who have spent a lot more time thinking about it than you, and who are probably smarter and better educated to boot, never figured out.
Guess what: it looks like the authors didn't control for income (or even just whether the person has a job!) or intelligence.
I just read the paper; there are many diet and exercise-related controls, and that's it as far as I can see. I totally agree with the OP that being the kind of person who might drink coffee probably has a confounding effect on health. I can't even think of how to try to control it properly.
I don't understand how you get from "most studies are inaccurate" to "but you shouldn't assume the ones that get reported on in the media are inaccurate in a way that we can detect". Why not? People who are performing studies that get picked up by popular media are the least likely people to be performing their work rigorously, and the most likely to be choosing a sensational hypothesis that's difficult to measure correctly.
I agree that we might eventually get to a point where the amount of skepticism and cynicism about scientific results vastly outweighs the number of papers published with poor experimental design or lack of real statistical significance. But I don't think we're currently anywhere close!
You make some good points, so I hope this doesn't seem dismissive-- my claim is not that the study is right. My claim is simply that the editors of the NEJM decided to publish the study having read it in greater depth than any of us here would care to, so if I read the CNN article and think there's an obvious reason it's not true, I can be pretty sure I'm wrong.
Now that being said, I think we're talking at cross purposes. The study found a link between coffee drinknig and mortality; I misread the GGP as being critical of that link, where it's actually just a hypothesis which would explain it. So none of this is really relevant :P
I read the "methods" section of the study, and saw no reference to controlling for occupation nor socioeconomic status (as opposed to marital status, self-reported health, and other things that are listed).
However, I only read it briefly, so I may have missed something.