In the first genuinely interesting development to emerge from the polling industry since the federal election failure, the leading name in Australian opinion polling has announced a significant methodological overhaul.
The announcement comes from global market research giant YouGov, which in December 2017 bought out Galaxy Research — the company that conducts Newspoll for The Australian. The parent company initially showed no inclination to meddle with Galaxy’s hitherto winning formula; it involved a mix of online and automated phone polling, and — in common with the rest of the industry in Australia — a rather lamentable lack of transparency as to how its raw data was processed into published results.
History now records that Galaxy’s “our record speaks for itself” defence for the latter deficiency went out the window with the May federal election result. The three sets of Newspoll numbers it has since produced for The Australian have not been taken seriously beyond the paper’s own editorial desk.
The announcement of an inquiry, by the Association of Market and Social Research Organisations, on the Monday after the election appeared to hold out the promise of a public airing of the industry’s dirty linen, but close observers are highly skeptical about its progress. While the inquiry has heavy-hitting academics and statisticians on board, neither YouGov Galaxy nor Essential Research is a member of the body responsible for it, and they have, at best, been partly co-operating with its work.
So it came as a breath of fresh air to learn on Thursday exactly what YouGov plans to do differently going forward: a move to “the standard YouGov methodology for national and statewide polling”, including an entirely online survey mode and the use of “additional variables for weighting such as education and more sophisticated regional segments”.
Better yet, the emphasis placed in the announcement on the pollster’s “values of transparency and methodological rigour” suggests we may soon enjoy the level of detail that routinely accompanies poll releases in the US and the UK, including publication of sample weightings and intricate demographic breakdowns for each poll.
For a clear sense of the significance of this, one need only compare the panoply of data offered by one of YouGov’s British poll releases with the threadbare detail it publishes in Australia.
Once that genie is out of the bottle, rival pollsters will have to either follow suit or abandon any right to be treated as serious competition to the acknowledged industry leader — particularly if anything comes of YouGov’s call for the establishment of a local equivalent of the British Polling Council, which imposes exacting standards on its members concerning the detail they must publish of their methods and results.
Of course, transparency alone will not be sufficient for the industry to recover the strong reputation it held until quite recently. That will require runs on the board in the form of more-or-less accurate pre-election polls, for which no opportunity will emerge until the Queensland state election still over a year away.
It’s far from certain that YouGov will prove able to get better results by dropping the telephone component of its polling, notwithstanding that phone polling is less conducive to the kind of detailed demographic parsing that it apparently has in mind.
Nonetheless, the movements the pollster records over time within demographic and geographic sub-samples will almost certainly offer insights into the shifting sands of public opinion, even if skepticism will remain as to how it sees the numbers combining in aggregate.
In the most optimistic scenario, polling may be on the cusp of striking a healthy and happy medium — robust and transparent enough to be worthy of more than a second glance, but relieved of the sense of superstitious awe that has allowed it to be weaponised by political plotters so often over the past decade.
How should we approach polling in 2019? Send your thoughts to boss@crikey.com.au. Please include your full name for publication.
The significant failures that I have followed relate to the multicultural dimension of the electorate – a segment that helped give the government its majority. Most polling companies fail abysmally in areas of NESB responses and have no strategy to recover. They cannot even tell whether they have anything like a representative sample by major ethnicities. They got it wrong on same sex marriage and a number of key elections. I’d very much want to see how they are going to correct for this dramatically White bias.
I don’t see why we should pay any attention to the polling companies after their massive failure. IMO because it has such an outsized effect on democracy, political polling should be tightly regulated. Perhaps only one published poll per three months for starters, a requirement to have the margin of error prominently displayed and reported, and the publication of all the pollster’s underlying methdology and assumptions. Otherwise they can piss off.