“Malcolm, you are hearing the interpretation of a highly qualified scientist and you’re saying, ‘I don’t believe that’ — is that right?” an incredulous Tony Jones asked of former One Nation Senator Malcolm Roberts during a recent Q&A episode. The then-senator had demanded incontrovertible proof of anthropogenic climate change from fellow panellist Professor Brian Cox, and made wild allegations about evidence having been manipulated by NASA, the Australian Bureau of Meteorology and others.

The former senator has made a name for himself challenging scientists and scientific institutions. Ostensibly he was holding them to a higher standard of evidence, although the same rigour clearly did not apply to his own, personal situation.

For years, Malcolm Roberts has been bombarding scientific institutions, individual scientists as well as journalists and parliamentarians with strong claims rejecting climate change research. He has also made severe allegations of scientific misconduct against individual scientists, which have been dismissed after due consideration.

At the heart of his rejection of the science establishment and its conclusions lies a vast climate change conspiracy, aided by both major parties as well as the Greens, reaching for “massive over-government”. Not confined to Australia, “The UN, on behalf of international bankers, wants to end national sovereignty and individual freedom”. As a result, “People in developed nations are being enslaved to international banks”.

The denialist traits in Roberts’ thinking have previously been documented. The cherry-picking of evidence, the impossible expectations raised (for instance in terms of predictive accuracy and scientific consensus), and the underlying conspiracy theories are just some of the hallmarks of denialism.

Similar to democratic politics, science can experience gridlock, detours and errors. Progress often is painstakingly slow. In recent times we have also become aware of a number of serious shortcomings in the scientific process.

In modern science, the process for establishing valid knowledge, or rather knowledge “warranting belief”, hinges on peer review and professional reputation. In recent times, the process has come in for sharp criticism on account of a number of practices undermining scholarship.

The pressure on academics to publish (“publish or perish” in the vernacular) makes them more reliant on the scientific institutions and their gatekeepers, such as journal reviewers, editors and tenure committees. These then direct researchers by effectively sanctioning the acceptable (and therefore permissable) research methods, research questions and available data.

The pressure to publish in order to secure tenure and promotion may also lead to an inflation of “Type 1 errors”. A Type 1 error (also known as a “false positive”) is the mistake of erroneously believing something to be true when it is not. Scientific journals tend to cherish surprising results. As a result, scientists may test more frequently for contrarian hypotheses and produce more (erroneous) chance results. As reported by Scientific American magazine a number of fields of scientific endeavour may be plagued by a significant proportion of false positives and exaggerated results.

Equally concerning, studies that fail to find novel results may not even be reported by the authors, effectively censoring the publicly reported evidence. A variety of fields such as medicine and psychology have in recent years been hit by an apparent crisis of reproducibility and experts have pointed to the elevation of a disproportionate number of false discoveries.

In the social sciences, some of these biases and problems may be even more egregious than in the natural sciences.

Overall, such institutional biases are casting a shadow over science, slowing scientific progress. But scientists are increasingly aware of these issues and gradually the scientific process is responding. Initiatives such as the appearance of “minimal threshold journals” and article retractions or campaigns to report all trials and results attest to the (albeit slow) self-correcting capacity of science.

Scientific knowledge is sometimes characterised as “warranted belief”. But what exactly warrants belief? What constitutes valid knowledge? How do we believe in the authority of a particular theory, given that few of us, even scientists, are in a position to verify them independently? We do so by trusting in the institutions tasked with producing valid representations of the world around us.

In light of the inevitable shortcomings of the scientific process, the scientific community as well as the wider community need to maintain a healthy degree of scepticism about scientific findings and scientific evidence. But scepticism can sometimes degenerate into paranoid denialism. Seemingly “common sense” explanations, or conspiracies can provide a degree or comfort when complex and less-than-perfect expert opinion abound.

In the words of philosopher Brian Keeley “Social trust is fundamental in making scientific knowledge possible. Without social trust — not blind, unwarranted trust, but trust, nonetheless — […] scientific progress screech[es] to a halt.”

Like democracy, science, and the scientific process, are flawed. But they are still preferable to all the other forms of generating valid knowledge that have been tried, including Malcolm Robert’s.