Science journalism: don't forget the editors

Continuing the current discussion of the questionable quality of popular science journalism, British researcher Simon Baron-Cohen weighs in at the New Scientist with his personal experiences of misrepresented research. Baron-Cohen complains that earlier this year, several articles on his work linking prenatal testosterone levels to autistic traits, including coverage in the Guardian, were titled and subtitled misleadingly:

It has left me wondering: who are the headline writers? Articles and columns in newspapers are bylined so there is some accountability when they get things wrong. In this case, it was a nameless headline writer who seems to be to blame. Did he or she actually read the journalist's article?

I've experienced this problem myself: I've actually written a press release and fact-checked it, only to find that the title was altered to something misleading after it left my hands. The change happened without my input, and I was unaware of it until the researcher I had interviewed contacted me (assuming, no doubt, that it was my doing). The problem was eventually fixed, but I felt that my credibility as a writer and scientist was impacted, and the researcher was understandably unhappy about the process. It was an unfortunate situation all around.

In journalism, titling is often divorced from the writing process - as a writer, you can suggest titles, but they will likely be changed by editors or other staff. This is not unique to science journalism, but it may disproportionately affect science pieces, simply because it is so challenging to briefly and accurately summarize a study's outcomes and relevance without resorting to jargon. Titles inevitably oversimplify the science. That's not a problem as long as the article builds on the title to clarify and explain further. But when the editors responsible for titling pieces don't understand the science or its context, the title can end up not merely simplified, but misleading, inaccurate, or just plain creepy. Bad titles are particularly problematic because the title frames the entire story for readers, and can predispose them to read everything in the article in a completely inappropriate light. Misleading photos, illustrations, or captions can have a similar effect. Yet these elements are not necessarily vetted by a qualified individual who understands the research.

Baron-Cohen works on autism. His work has highly emotional implications for parents of autistic children, and is thus particularly vulnerable to sensationalization by unscrupulous or incautious editors. He suggests that such misrepresentations can cause serious harm to the public:

Scientists are rightly regulated by ethics committees because they can do harm to the public. The media too has the potential to do harm. Should there be some similar before-the-event regulation here too?

This is an intriguing question. Who, in fact, holds the media accountable for accuracy in science journalism? Often, criticism of bloggers centers on their lack of accountability to editors or formal media outlets. Traditional journalists are seen as more accountable, both because they have others checking their work, and because they have an incentive to maintain their own professional credibility and the credibility of their organization. Yet even within the framework of traditional media, mistakes are clearly made. And the recent debate about scientists vs. science journalists as communicators obscures the fact that many mistakes don't originate with either the scientist or the journalist, but with editors and others involved in the publication process. Unfortunately, when such mistakes or misrepresentations are eventually identified and corrected, the revised message may not seem substantially different to the public, and may never get as much attention as the original error. (You never get a second chance to make a first impression, right?)

So can linking scientists directly with the public help? One of the benefits attributed to science blogs is that researchers may be able to both explain and frame their work more accurately than a media outlet peopled by generalists and editors looking for catchy ledes. I think that by providing direct access to scientists, science blogs can effectively complement, though perhaps not replace, science journalism. Several of our colleagues are currently working to get scientists more involved in communications in the blogosphere and during graduate school.

In Baron-Cohen's case, the Guardian stepped up and let him publish a clarification of his research. His key point - that hormonal levels predict autistic traits, not autism per se - was a subtle one, and not surprisingly, it didn't seem to get as much attention as the original piece. The comments on his clarification represent varying degrees of disbelief and ire, including accusations that autism scientists are deliberately concealing and misrepresenting their findings. Ugh. Reaching out directly to the public might have worked better if it had happened before the original article framed the discussion in an emotionally charged manner.

Just last week, Ben Goldacre took the British press to task for misrepresenting new prostate cancer screening research - for, in his words, choosing to "ignore one half of the evidence and fail to explain the other half properly." The Guardian's own article on the topic was titled "Prostate cancer screening could cut deaths by 20%." That's technically correct (for men between 55 and 69). But as Goldacre points out, the original NEJM research article is equivocal about the risks and benefits of prostate cancer screening with this particular technique. To prevent a single prostate cancer death, 1410 men would have to be screened and 48 men would have to be treated. Treatment carries the risk of serious complications like impotence, and there is a high rate of false positives with this test (as much as 50%). The Guardian article (which has the same byline as the piece on Baron-Cohen's autism research) mentions this risk - but only in the very last two paragraphs, where it does very little to counterbalance that Pollyanna-ish title. Headlines from the WaPo ("Prostate cancer screening may not reduce deaths") and Sydney Morning Herald ("Prostate cancer blood test does little to decrease death rate") are more consistent with the authors' conclusions (and that of a second new prostate cancer study, which the British press inexplicably neglected to mention).

As Goldacre explains, oversimplified and misleading headlines - like "Prostate cancer screening could cut deaths by 20%" - do the public a disservice:

For complex risk decisions such as screening, it has been shown in three separate studies that patients, doctors, and NHS purchasing panels make more rational decisions about treatments and screening programmes when they are given the figures as real numbers, as I did above, instead of percentages. I'm not saying that PSA screening is either good or bad: I am saying that people deserve the figures in the clearest form possible so they can make their own mind up.

But in order to find the "real numbers" in this study, along with the authors' own interpretation of their work, you'd have to go to the NEJM article itself - and the Guardian article doesn't link to it. It's surprisingly common practice for press releases and articles to fail to cite the source paper, or even name a lead author - a reader often has to go to the journal's website and search for the relevant article using keywords. In the intertube age, it's downright ridiculous that media outlets can't just paste in a URL to the original article. Both Goldacre and Mark Liberman at Language Log have called for explicit links from media coverage to the peer-reviewed articles being discussed. This might help short-circuit the propagation of error that happens when a mediocre press release is processed to yield a worse brief article and a hopelessly inaccurate sound byte, without any of the authors apparently understanding the original research. At the very least, it would give a concerned reader somewhere to look for more information.

Liberman says,

What Ben is suggesting, I guess, would be something like Google News for science, with the addition of links to the underlying scientific publications (if any), and to coverage in blogs and web forums. Also, it would be useful to have the clusters of links be durable -- i.e. available across time, unlike Google News -- and perhaps linked into a loose network of higher-order relationships.

Coverage by bloggers is not necessarily any better -- it may often be worse -- but at least you have a shot at finding someone who has read the paper, not just the press release, and who knows the field well enough to understand the paper and to offer an independent interpretation.

To this end, Adam Bernard has created The Science Behind It: "The site was built to deal with my frustration at journalists summarizing scientific papers without citing their sources. It tries to infer from the details that they do include the probable candidate articles." In a similar vein, Dougal Stanton contributed "Just Fucking Cite It" , an all-purpose link to use when calling out articles that totally fail to represent the science accurately.

So will pushing science journalists to include proper citations in mainstream articles help? Well, the PLoS One community blog discussed their take on the problem Thursday:

PLoS ONE articles often receive a lot of coverage in the media and because our open-access articles are freely available to read online as soon as they are published, we always encourage journalists to link to the original study in online versions of their reports. However, with some notable exceptions, this doesn't seem to happen very often (although increasingly, links are being added to the PLoS ONE homepage where we often highlight newsworthy articles for interested readers). Of course, the reporters to whom we send our press releases (which include the URL for the articles) aren't always the people who are able to add the URLs to the stories with various web teams and editors often involved in the process of getting a news story online.

It all comes back to the editors, doesn't it? Regrettably, in discussions of the mainstream media process and how to improve science journalism, editors and other staff are often overlooked. But although it's necessary to have both scientists who can communicate effectively and journalists who can understand and explain the science, it's not sufficient. You also need an editorial team who won't screw the whole thing up!

A media outlet needs the right people in place, or the right process in place, to check the accuracy and objectivity of details like titles, layout, pictures, illustrations, captions, and citations. A media outlet under acute financial pressure to downsize, or one that prioritizes sensational headlines over accuracy in science reporting, may not have those safeguards.

While blogs have their downsides, at least there is usually just one person responsible for what you're reading. In some ways, that gives bloggers an advantage over mainstream journalists, who may have very little control over the way their piece is edited and presented. First impressions count, and in mainstream science journalism, editors have a lot of control over the way the science appears to the public - sometimes more than the scientists or journalists themselves.