Alice Walton demonstrates how you should report on the crappy public health story of the week

Sue Dynarski links to a typically awful news article on a correlational public health study, along the lines of the “chocolate prevents cancer!” garbage that inspired this blog’s name. This one is about a claimed link between coffee and early mortality in people under 55. What’s striking about this case is that I had already bookmarked an article by Alice Walton on the same study that does a great job of presenting the results for a mass audience.

Here’s what Alice gets right:

1) Emphasize the preliminarity and uncertainty of the results. Walton does this consistently, from the very beginning:

Those of us under 55 who drink a lot of coffee – more than four cups per day – may be at greater risk of an early death. [emphasis added]

2) Put the study in the context of the existing literature. She points out that these results are inconsistent with the “mishmosh of coffee studies all pointing at different outcomes”

But perhaps most uplifting of all is to remember that findings from a number of earlier studies contradict the new one and suggest that coffee is actually, at least on average, good for us. In fact, one recent study in The New England Journal of Medicine, following some 400,000 people, suggested that drinking up to six cups per day is actually linked to reduced mortality from all causes – 10% for men and 15% for women.

3) Talk about the theoretical reason for the reason – or lack thereof:

One problem is that no one really knows what mechanism/s could explain the coffee-death link. Some are candidates, however: There’s coffee’s ability to boost epinephrine (adrenalin) levels in the body, its inhibition of insulin function (though this is controversial), and the fact that it may raise blood pressure and homocysteine levels, which are both known to increase heart risk (though since heart disease was not increased in the study, these seem less likely).

4) Distinguish between simple correlations and actual measurement of causal relationships. Chip Lavie, the study’s author, is totally up front about this. He’s not really an outlier, either – most researchers who do this kind of work realize that their results are a guide to future research rather than established, incontrovertible facts. The real culprits in overselling these correlational results are university PR departments.

Also keep in mind, the current study only points to a correlation, not cause-and-effect. And it only measured coffee consumption at one time-point, not many throughout the years. There could be a lot of other things at play. “It is impossible to know if this association is causal or just an association,” says Lavie, “so one does not want to over-state or over-hype the dangers of drinking more than 28 cups per week, although I personally will make an effort to keep my cups at 3 or less most of the time.”

5) Take the effect size with the crazy huge magnitude out of the lede, if you mention it at all. Dynarski points out that the 20%-50% range cited by the US News article on the coffee study is completely implausible:

Here is a sniff test of the magnitude of this estimate: a similar, correlational analysis showed that light smoking (less than half a pack of day) is associated with an increase in all-cause mortality of 30%. Heavy smoking (more than half a pack a day), an increase of 80%.  These magnitudes are in the same ballpark as the coffee study, which immediately suggests to me that the coffee estimates are absurd.

To Walton’s credit, she puts that number deep in the article’s text and caveats it heavily. The news media needs more science writers like her, and much less of just copy-pasting official press releases and adding in some hyperbole at the top.