Fix the Nobel Prize — before it breaks science

Science is in crisis and all its sacred cows are being examined anew.

The problems began with psychology. A so-called “crisis of replication” was revealed, beginning in 2011. Hundreds of researchers worked hard to repeat 100 notable studies. While 97% of the originals had “significant results,” only 36% of the repeats did.

What had been discovered, published in reputable journals and widely cited could not be shown to be true again.

More than half of all findings were basically bunkum, including some that had travelled very far. Perhaps the most famous such example is power poses.

This was a study that got far more attention than its sample size deserved. I remember vividly watching the TED talk (which has now racked up over 9 million views) and being blown away.

The basic theory of power poses was that we can change our self confidence by posing our body confidently. Putting your hand in the air or standing tall with feet wide apart was supposed to raise testosterone and lower cortisol “and perhaps to actually improve confidence and performance in situations such as interviewing for jobs,” the study’s authors wrote. It all sounded very plausible. But, in 2015, a repeat of the original study found no such effect.

The repeat study had 200 participants. The original, just 42. So, all that arm-hoisting before all those job interviews was pointless.

The reasons for the crisis of replication in psychology were many. They included small sample sizes, problematic statistical techniques, funding dependent on finding results, and bias from journals who decided what to publish.

[Nobel Prize does not a presidency make]

It was not long before the rest of science admitted it faced the same problems. The editor of perhaps the world’s pre-eminent medical journal, The Lancet, wrote that science had taken “a turn towards darkness”:

“The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

One of the most compelling problems in science is the pressure for results.

Journals rarely publish studies that find nothing. If you spend years gathering data and discover none of it fits, the hypotheses you were testing remain unproven. Journals are not inclined to publish that.

And of course, universities will not hire academics who are unpublished. Grants will not be provided to scientists who have made no discoveries. Etc.

In short, the pressure to get results is extreme. As post-doctoral researcher Joseph Hilgard, surveyed by US news-site Vox, said: 

“This causes a conflict of interest: The scientist is in charge of evaluating the hypothesis, but the scientist also desperately wants the hypothesis to be true.”

The fixes proposed are many — funding laboratories rather than studies; rewarding good experiment design, not good experiment results; pre-registering studies so ones that found nothing can’t be hidden away; scrapping scientific journals and publishing findings online in real time.

All these are good ideas — some are a matter of urgency. But will they be enough? The system of rewards for positive results goes to the very heart of science.

Each year the Nobel Foundation awards prizes in physics, chemistry, medicine, literature and peace. (There is also a related prize for economic sciences). A Nobel Prize consists of a solid gold medal, a diploma and 8 million Swedish krona (A$1.2 million).

Nobel Prizes are rarer than Olympic Gold medals, rarer than Academy Awards. Their recipients are venerated, idolised and adulated. At the University of California’s Berkeley campus, special car parks are set aside for the tiny handful of staff that are Nobel laureates.

But Nobel Prize winners are simply brilliant researchers whose hypotheses turned out to be true. At the same time, it is likely many equally brilliant scientists ran equally good experiments, but never quite found the data needed to reject a null hypothesis. (A null hypothesis usually represents the status quo belief about the world.)

Nobel Prizes have other effects on science. The dream of winning one inspires young people to become scientists, and keeps many a scientist in the profession even while the pay is bad, the hours long and the job security poor.

Prizes are a good value way to motivate them. Very cost effective. And of course, Nobel Prizes are obviously not the problem with science per se. But they are by far the most prominent manifestation of the tendency to reward results rather than good practice.

A wholesale change to science might include — just as a suggestion — the establishment of an even larger prize for exemplary scientific practice in replicating studies. Or a special, even larger prize for running dozens of high-powered experiments on highly plausible hypotheses and coming up with nothing.