Media reporting of health offers fertile pickings for researchers.

One of the latest critiques comes from English researchers who rated the quality of scientific evidence behind dietary health claims reported over one week in November 2008 in the top 10 selling UK newspapers.

It is said to be the first “systematic evaluation of dietary health claims in the media”. Of 111 such claims identified, 72 percent and 68 percent (depending on which ratings criteria was used) had levels of evidence lower than the convincing or probable categories that are recommended for dietary health claims.

The Guardian’s science columnist Ben Goldacre, who was one of the authors, says the results suggest “that the majority of health claims made, in a large representative sample of UK national newspapers, are supported only by the weakest possible forms of evidence.”

The abstract is freely available from the journal Public Understanding of Science (Croakey will send the full article to readers who request it – just leave your details as a comment or email me direct).

The researchers wrote:

Newspaper journalists commonly source news from individuals and organisations seeking to raise  awareness of a single issue or event. To this end, many press releases are produced by public interest groups, trade bodies, public relations companies working for industry, government departments, universities, journals and charities to broadcast their respective views and advice, and these are often reproduced uncritically.

Since the science of health claims is of interest to the general public, relevant to health risk behaviour, and an area where the public are often misled, then critiquing misleading coverage may be an appropriate subject for school-age education, and an appropriate target for health promotion activity.

So how do journalists and the media respond to these sorts of critiques?

I asked Gary Schwitzer, publisher of the US-based HealthNewsReview.org, for his experiences. As mentioned in this recent Croakey post, I was struck by how little defensiveness was apparent among journalists when Schwitzer delivered his critique at a recent Association of Health Care Journalists conference in Philadelphia.

Most seemed genuinely receptive and keen to learn how they could do a better job of serving their audiences.

But this is not always the case, as Schwitzer explains below.

***

Defensiveness is a barrier to quality improvement

Gary Schwitzer writes:

In response to criticisms of media coverage of health matters, I have heard journalists make comments such as:

  • I don’t get my inspiration from news releases.
  • The criticisms are highly generalised.
  • Why don’t you look at a journo’s body of work rather than rate individual stories?

I agree that generalisations are not helpful and it is important to acknowledge the spectrum of news quality. To avoid making sweeping, generalised criticisms, the Media Doctor sites and ours apply specific, standardized criteria every day in the same way to all stories reviewed. Specific comments are posted to back up our grading on these criteria.

We are seeing simultaneously some of the best health journalism ever and some of the worst.  A few lines like that seem to squash thin-skinned defensiveness. But they’re not just throwaway lines meant to placate. It’s true and it’s an important context to remember when you’re engaged in media criticism.

But it is not uncommon for journalists who are doing quality work to react defensively to criticisms that are clearly not relevant to their work. It is almost as if when you malign one in our union, you malign all of us.

It is a fact that we have seen a growing trend of stories that rely solely or largely on news releases – and from some leading US news sources.

The “body of work” comment is one I’ve heard from some of our harshest critics – from some people who’ve probably written us off.

But how do you argue with the cumulative “body of work” overview of probably more than 2,000 stories that MediaDoctorAustralia has documented and the “body of work” overview of more than 1,500 stories that we have now published?

It may not say anything conclusive about any one journalist’s performance.  That is not the intent.

But it speaks loudly and clearly and indisputably about the performance of big picture health journalism.  The scores have remained amazingly consistent over time in our 5 years – that nearly 70% of stories fail to quantify harms, benefits or discuss costs.

The harms and benefits issue is exactly what Moynihan and colleagues reported in their groundbreaking 2000 NEJM paper (Moynihan R, Bero L, Ross-Degnan D, Henry D, Lee K, Watkins J, Mah C, Soumerai SB. “Coverage by the News Media of Benefits and Harms of Medications.” New England Journal of Medicine 2000; 342(22):1645-1650.).  This is a body of work that cannot be denied.

It should force any journalist to reflect on what he/she has done within that broader picture.  If, after doing so, one can say, “I’ve done the best I can.  I don’t think I’m guilty of these flaws” – then so be it.

Congratulations.  We hope you can join us in the rising tide to lift all ships.

None of us takes criticism well. But some journalists, who can dish it out so well, seem particularly incapable of accepting criticism from any corner.

This is not a step towards quality improvement. Indeed, it may be a sign of a growing disconnect between some journalists and their intended audiences.  If there can’t be a dialogue, if journalists can’t accept feedback and criticism, is it any wonder that users move on to other media sources and formats where such a dialogue can take place?

My wife, a nurse, compares how her profession operates with how ours does.  There isn’t a shift that goes by without nurses having to account for their performance that day – a review of what went right and what went wrong, how to address errors and how to improve in the future.

How/when does that take place in most newsrooms? Indeed, the industry we cover has lessons for us about quality improvement.

Finally, some of the journalist pushback we’ve received is, in my view, a self-indictment of the journalists – an admission that they are performing stenography more than journalism.

We’ve had journalists say to us:

  • I won’t include harms, absolute risk/benefit data, NNT, even conflicts of interest if researchers don’t provide that info;
  • “Don’t fault me for reporting on presentations highlighted at conferences.  It’s a competitive world.  Fault the meeting organizers for highlighting marginal reports.”

In other words, ‘I only report what I’m given’. If these attitudes were shared with readers, circulation would plummet.

There’s also a blame game going on there:  don’t blame me, blame the researcher or blame the journal or blame the conference host.  That is an incredibly immature, irresponsible, unprofessional attitude.

It is time for journalists to accept the responsibility that comes with covering these vital topics.

I urge readers to examine the Statement of Principles of the Association of Health Care Journalists, which I wrote, and which AHCJ endorsed and is now proud to promote.

I believe you’ll find all of the criteria employed by Media Doctor and HealthNewsReview.org embedded in that statement.

Gary Schwitzer at Association of Health Care Journalists conference, Philadelphia

Melissa is correct in her observation that there was very little defensiveness from journalists at the AHCJ conference in response to my critique.  You would expect some in a room of 80-90 people.

There was a similar silence when I did a similar workshop before a similar-sized crowd at the AHCJ conference in Chicago last year.

Part of that may be due to the fact that I’ve been out talking about this so frequently in so many venues for the past five years. I’ve travelled nonstop the past two years giving talks and workshops – all of it related to HealthNewsReview.org.

The more opportunities I have to explain our motivation, to explain that we’re not media-bashing, that we are truly concerned about the misinformation provided to so many consumers, that there is true HARM in haphazard health care reporting….the more we win converts.

***

And what about Australian journalists?

Are we interested in constructive critique? Or do we batten down the hatches?

MediaDoctorAustralia has just posted some of the feedback they’ve had over the years from journalists. Some find the site’s critique useful, others do not.

It would be nice to see more media organisations making use of the work done by outfits like HealthNewsReview.org and MediaDoctorAustralia – would it be asking too much for them to update the online version of stories with links to such reviews and critiques?

This would provide a useful service to readers, for very little cost. I’d be willing to lay a bet that it might also build some loyalty and appreciation amongst their audiences.

And finally, on a related note, is a recent piece by Tony Richardson, who works in advertising.

When I first saw it, I thought – aha, this is something I could post at Croakey to ask the question of whether the health care sector could learn from McDonald’s…

But perhaps it’s more relevant to the media industry, given that health care has done so much more work on quality improvement and open disclosure, at least compared with my industry.

***

Confessing your mistakes can be good for business

Tony Richardson writes:

The three worst coffees I’ve ever had, in no particular order:

  • British Airways London to Madrid – As nasty as the hostie serving it.
  • Gundagai Café – It could have been tea, coffee or hot Murrumbidgee River water. I really couldn’t tell.
  • McDonald’s.

Unlike the Gundagai Café or British Airways, I will consider trying McDonald’s coffee again. Why? Because they said sorry.

In a rather surprising advertising campaign McDonald’s are apologising for the sub-standard coffees they have produced in the past.

It seems research showed that many Australians found their coffee convenient but … dreadful. I certainly have.

Where many marketers would have simply looked at the dud research and claimed even more loudly that their product was the superior in every way, McDonald’s has chosen to face the music.

According to a rather nice long copy print ad (remember them?) they are now training baristas and offering a replacement cup if you don’t like the first one.

McDonald’s have turned a negative into a positive at almost no cost to themselves.

As a result consumers listen. We actually quite like an organisation that admits it was wrong and outlines how it will fix the problem. It’s so unusual.

I think Lipton Tea did a similar thing when it changed its name and packaging and sales temporarily crashed. What is it with hot drinks?

Anyway, if your consumers are angry about something, consider apologising. It could turn sliding sales into booming sales.

TIPS FOR MARKETERS – ‘Spin’ is so prevalent and honest confession so rare, that apologising for a mistake will almost certainly improve your brand’s stature.