Evidence-based policy is all the rage these days. Scarcely a moment goes by without it slipping into some utterance by a politician, bureaucrat or lobbyist.

And it sounds so sensible. Of course we want policies based on evidence, don’t we? Otherwise they mightn’t work, and our money would be wasted. Worse, we might end up with policies that do more harm than good.

But even a casual look at the history of evidence-based medicine, which some would claim as the mother of this latest new religion, reveals grounds for scepticism.

Here are a few:

Rhetoric is easy: Canadians coined the term “evidence-based medicine” in the early 1990s and it has since mushroomed into an international push to improve health care. But the medical journals are still full of studies about patients missing out on useful treatments, or having treatments whose benefits are unlikely. The reasons for the difficulty of translating the rhetoric into reality are far more complex than simply “doctors behaving badly”, and involve a multitude of factors. If it’s so difficult to achieve evidence-based health care, then it will be a million times tougher in the policy arena, where there are arguably far more barriers to surmount. As a UK policy buff once noted: “When votes are at stake, evidence goes out the window.”

Irrational assumptions: It’s a fatal mistake to assume that policy is a rational process in which you identify a problem, work out the best evidence-based solutions, then implement them. In the real world, the process is messy, chaotic and far more focused on solving political problems. Policies are so often developed to distract the punters from the killer headlines on page one. Who has time for evidence with deadlines like this?

What evidence?: You can’t have evidence-based policy without evidence. And you can’t have evidence without a willingness to invest in the sort of research that might help answer pressing policy questions. Australia has not got a brilliant track record in this respect. We do, on the other hand, have a brilliant record of suppressing findings that might usefully inform policy by highlighting problems with government programs.

What debate?: Rigorous public and professional debate is crucial to the development of evidence and its integration into policy. This is sorely lacking – so many researchers and policy makers are reluctant to speak up, fearful of upsetting their funders or political masters. So it’s not surprising that much media coverage reveals a poor understanding of evidence-based concepts. That the media would critique Rudd’s idea for one-stop childcare centres as “copying” overseas programs is bizarre. Surely the fact that there is some evidence from overseas to support a policy is a plus, not a minus? Nor does the mindlessness of so much political debate – the government says one thing and so the opposition must say the other – allow much space for what the evidence says might work.

Whose evidence?: Anyone with a barrow to push makes sure it is laden with “evidence”. The term is so widely and indiscriminately used, it has become meaningless and is often invoked in support of all manner of dubious causes. The public, the media, and even policy makers themselves are often not skilled at discriminating between evidence which is reliable and useful, and that which is not. Even when there is evidence to show that an intervention works in one setting, this does not mean it will work elsewhere. Evidence has limitations aplenty.

Behind the times: Compared with North America and the UK, Australia has been relatively slow to set up the structures and systems that might support the better integration of evidence into policy. The upside of this is that we can learn from their mistakes over the past decade or so. One clear lesson is that they are now more likely to talk about evidence-informed policy, acknowledging that many other factors also influence the policy process. Evidence is only one player at the table and often lacks the strongest hand.

It’s not that I want to discourage talk about the role of evidence in informing policy. It’s just that I can’t see it really making a difference unless some of those chanting the mantra of evidence-based policy start to put some real muscle behind it.

Declaration: Melissa Sweet has written about evidence-based policy for various health groups in Australia and overseas, including the NHMRC.