The news percolating across the Internet a few months ago that chocolate can help you lose weight turns out to be a hoax. Science journalist John Bohannon created the study and reported the results in order to expose how easy it is to get junk science into the media.
The study was real: a randomized, controlled study of 16 people that found the subjects who followed a low-carb diet and ate a 1.5 oz. bar of dark chocolate daily lost weight faster than the control group that dieted but consumed no chocolate. The problem was that the study was intentionally designed to be flawed: a small sample, too many uncontrolled variables, and thus results that lacked statistical significance.
Bohannon sent the study to a “pay-to-play” journal that uses no peer review, and conjured up a big press release to trumpet the results. Reporters picked up on the story and reported it without consulting independent experts or researching the quality of the study—just bad journalism.
This is a growing problem especially in the field of health or nutrition research in which the public is intensely interested, especially these “too good to be true” stories. They inevitably get lots of clicks and viral promotion in social media. When journalists don’t do their job by assessing the quality of the research, the public is massively misled. Of course, scientists are not off the hook for their shoddy experimental design, but it’s part of journalism’s job to keep them honest.
Bohannon has been criticized by many because he also misled the public by creating and publishing a study that is intended to mislead. Here is one such critique:
Bohannon and his colleagues decided to create a wrong to prove that wrongs exist. They lied to the public to make their point. Granted, it’s unlikely that anyone wll be harmed by eating more dark chocolate. But not only does the caper do a disservice to people who are desperate for meaningful information about health and nutrition, it also undermines all of science and all of journalism. There’s real wrongdoing in both science and journalism (most infamously, see Stephen Glass, Jayson Blair, Janet Cooke, Jonah Lehrer, Brian Williams). But intentionally creating wrong to make a point is both bizarre and potentially very damaging.
But I don’t quite buy this criticism. Bohannon is a whistleblower, informing the public about wrongdoing in his profession that causes serious harm to the public. Generally speaking, whistleblowing is permissible (and perhaps obligatory) when the whistleblower has good reasons to believe informing the public will prevent harm at a reasonable cost. It is true that many people were misled by Bohannon’s study, but it is on a rather trivial matter that is unlikely to cause substantial harm. The harm of lazy journalism is very serious and responsible journalism is one of the checks we have against irresponsible science, which has the potential for enormous pubic harm. My only reservation with this utilitarian justification is whether Bohannon’s hoax will actually prevent harm. Will it have an effect on those lazy journalists who publicized this study without checking its validity? I doubt it. It probably will be swallowed up in the deluge of new problems and new scandals that populate the next news cycle and will be forgotten long before anyone is held accountable.
Bohannon can’t be blamed for being a single voice in the wilderness. He wasn’t justified because of the harm he prevented. He was justified because his actions exhibit care for the practice of journalism and its future, unlike the stenographers who copy press releases and call it journalism.
The problem is much larger than a single whistleblower can solve. Society’s incentive structure is misaligned. When scientists and journalists are encouraged by the institutions in which they work to pursue profit rather than doing quality work, this is the result. When success is measured in dollars, its dollars you get, not quality.