The three major political pollsters all got it badly wrong on May 18. Newspoll’s election-eve poll predicted a two-party-preferred (2PP) result of 51.5%-48.5% to Labor, almost exactly the reverse of the actual result, with the Coalition’s primary vote more than three points higher than polling suggested.
Galaxy Research, also controlled by YouGov, had Labor winning 51%-49% and underestimated the Coalition vote by two points. Ipsos also predicted 51%-49% in its last poll. Essential reported 51.5%-48.5% in its final poll. Days earlier, veteran polling outfit Roy Morgan also weighed in, predicting 52%-48%.
To be fair, the reason the 2019 failures stand out so much is that Australian pollsters have been enviably accurate in the past compared to the UK and the US — right up to 2016. But as ANU Vice-Chancellor and Nobel Prize-winning astrophysicist Professor Brian Schmidt, who knows a thing or two about numbers, pointed out the day after the election, the confluence of polling results in the 2019 election was improbable — highly improbable.
Psephological site Mark The Ballot made a similar warning, presciently, before the election. But as former Essential pollster Andrew Bunn pointed out to INQ, what made it really weird is that the same wrong result was achieved by pollsters using very different methodologies. Newspoll, owned by British company YouGov via its ownership of Galaxy Research, is a combination of robocalling and online polling. Essential uses online polling, while Ipsos, which appeared in Nine-Fairfax papers, used phones. But all got it wrong.
A review of the 2019 election polling failure is now being conducted for the Association of Market and Social Research Organisations by an array of industry luminaries.
So, what happened?
A number of explanations and theories have been advanced to explain what happened in May — some by the pollsters themselves, others by those prepared to be more sceptical about their methods:
- A late swing to the government. This explanation is easily dismissed — the swing must have been so late it happened after the close of voting, as Galaxy had an exit poll on election day showing victory to Labor by 52%-48%. But it’s worth noting pre-poll votes heavily favoured the government — the result of votes cast on May 18 itself were much closer to what the polls suggested. Galaxy’s exit poll may have been the most accurate of the campaign, but it missed the early swing to the government — in the third of voters who voted before May 18.
- Shy Tories. As the name suggests, a British phenomenon, coined in the 1980s when Margaret Thatcher (and, later, John Major) racked up wins despite apparent lack of support. The problem, as Bunn noted to INQ, is that for some reason they’re shy in some elections but pretty unabashed in others — no one underestimated Tony Abbott’s support in his 2013 landslide win.
- Herding. There’s been a lot of discussion about whether pollsters, worried about being out of step with other polls and especially the most influential, Newspoll, adjusted their weightings. Pollsters have a huge range of weighting methods, which adjust the raw data to better reflect the demographic features of the actual population compared to the sample, to choose from — and can thus use them to bring their results more into line with what they were seeing elsewhere. This might explain why the same problem occurred across polls with very different methodologies. William Bowe warned Crikey readers a week before the election to beware of this exact possibility given the strange uniformity of the polls. But as Tasmanian psephologist Kevin Bonham points out, there’s no actual evidence that this happened.
- Preferences and don’t knows. Pollsters have to allocate the preferences of those who indicate a voting intention for a minor party, and use historical allocations as a basis. But what if those allocations no longer apply? The LNP collected a significantly greater proportion of One Nation preferences in 2019 than in 2016. And results from respondents who answer Don’t Know are traditionally allocated using the results of the overall sample. But what if, due to the nature of the campaign, the most disengaged and least informed voters broke persistently for one side rather than being evenly distributed?
Others have approached the problem from a different perspective — particularly Bonham, Mark the Ballot, and Simon Jackman and Luke Mansillo of the University of Sydney. All have tried to mathematically adjust polling data over the last parliamentary term to reflect what we do know about what voters actually felt: we have the election results from 2016 and 2019, which tell us that the polls were very accurate in 2016 but went badly off-beam at some point before the 2019 election.
The question thus becomes how and when they went off-track. The answer — which requires an excellent understanding of maths — depends on the assumptions you make, however Jackman and Mansillo suggest the Coalition vote began recovering from late 2018, but the recovery wasn’t picked up by the polls. Mark the Ballot‘s models suggest either the Coalition’s vote was understated by polling throughout the parliamentary term from 2016-19 but the problem got worse as the election neared, or it enjoyed a late swing in the lead-up to the election that was missed by pollsters. Bonham, who has an excellent analysis of the different approaches, advances the hypothesis that Scott Morrison’s arrival as Prime Minister in the wake of the ousting of Malcolm Turnbull was the catalyst for an improvement in the Coalition vote that the polls failed to fully pick up.
While there’s no hard evidence for Bonham’s hypothesis, it would explain why Labor did well in the Super Saturday byelections last year against Turnbull — especially in Queensland, where it comfortably retained Longman, and Tasmania, where it retained Braddon — seats they easily lost in May against a different Prime Minister. Remember that then-LNP president Gary Spence urged the Coalition to ditch Malcolm Turnbull last year because of his lack of appeal in the sunshine state.
But there are so many variables with this scenario — and knowing when the polls went off track doesn’t explain why. As Jackman pointed out to INQ, the failure “could have been driven by any one of real changes in public opinion, biases in polls, or changes in poll biases — perhaps it was when YouGov bought Galaxy?”
Jackman — who is now working with Mansillo on further work on the election for the ANU — thinks herding is a serious issue in the polling community but it’s almost impossible to verify unless the pollsters themselves reveal it. Bunn thinks that, despite the Association of Market and Social Research Organisations (AMSRO) inquiry currently underway, we’ll never find out exactly why all the polls were off — because pollsters themselves might not know.
If so, that leaves those who rely heavily on polling in their professional lives with hard choices.
These are superb deep dives. The media series just completed and now this one do that rarest thing: take a complete step back, jettison all assumptions about how our information landscape functions, and look with fresh eyes ‘ponst it. It is absolutely critical, because our long-unquestioned received information frameworks – ‘Journalism’, and (here) ‘Large Scale Opinion Polling’ – are simply no longer fit for purpose (if they ever were). The consistent causal factor in both these epistemologies now being ‘broken’ is the underlying smashing of the lop-sided compact-by-default of the mass media era – that is, a tiny information cohort’s capacity to artificially leverage a mass audience ‘grand narrative’ onto and/or from a disparate mass of people, based mostly on a technical happenstance of history – by the Internet.
The harsh reality is that like ‘journalism’ (as opposed to reportage) ‘Polling’ is a profoundly bullshit contrivance. If I ask 100 people ‘What do you think about X, and what is that going to make you do about Y?’ here are the epistemological problems simply off the top of my head:
1. They may have no opinion at all but provide you one because…you asked. (Most common)
2. They may think they think A but really think B. Or think A, but for ill-examined reasons that make them do the opposite of what A might suggest to polls (ie I am deeply worried about the disruptions of climate change, but express it as a fear of ‘change’, so I will vote for fossil fuel stability.)
3. They may tell you one thing but change their mind tomorrow. Or in two minutes.
4. They may lie because they’re fed up with polls.
5. They may donkey-poll to get rid of you fast.
6. Or want attention/company and shape their responses to maximise engagement.
7. Those who try to answer accurately and comprehensively will by definition be a self-selecting outlier minority.
Then, and only then…do all the actual conceded ‘methodological errors’ come into play,
Why does the epistemological illegitimacy of mass polling – as absolutely opposed to hard statistical data/fact collation – as a democratic tool matter now? For the same reason ‘journalism’ (as absolutely opposed to reportage) is now dead (or needs to be slaughtered, for own sake): the rapidly evolving capacity of the internet to provide us all with actual 1:1 scale, real-time, fully comprehensive information pictures of human actuality.
In short: data analytics.
What political party ‘needs’ a bolted-on ‘poll view’ of the electorate when you can laser target the actual hard data; narrow your winning electoral strategy down to, say, seven or eight highly focused, deeply under-the-radar mini campaigns in carefully ‘electoral herd-seperated’ seats, that will swing just enough seats/college votes to game the whole herd-system over the line? Those mini-campaigns can be eight entirely contradictory ones, hermetic and ruthlessly pragmatic: Wei Chat spooking in Mandarin of 2786 known anti-SSM voters in Reid; slickly meme-ironic duchessing via Insta of 1457 known Mardi Gra attendees in Wentworth; Akubra tweets to the outer booths in Indi, college humour ones to the urbanites…etc. All the while, the old-fashioned mass meeja era at-scale ‘polls’ trying desperately to contrive trends/swings/patterns out of humanity’s messy irrationality, mercuriality and disparity…simply provide a useful distraction/smokescreen, while you ferret away, winning your national election by influencing no more than maybe 40, 000 acutely identified, highly tactical votes, tops.
Of course, having won it in such a deeply ‘meaningless’ way (in conventional political commentary terms)…you then turn to ‘journalism’ – those battalions of failed novelists who want to ‘craft a narrative’ out of the epistemological grey zone between strait-jacketed data and fully-unshackled imagination…- step in and ‘explain’ exactly Wot It Woz Wot Wun It….and what that means about ‘what Australians think’.
The internet is killing this mutual con, thank Christ. We aren’t falling for it any more, simple because it – and journalism – isn’t working any more. Neither is telling us (enough of) what we each know to be true about the world we live in. Or can find out for ourselves online. Teh Internet is breaking polling, just as it is breaking journalism. (But not in the bad way the professionals claim. It’s a liberation, not a destruction, of information that is happenin’ here. Even if it ain’t exactly clear yet..)
It’s why information professionals who aren’t clearly defined in epistemological terms – unlike the professional historians, technical policy experts and tenured academics, beat reporters and field-specialist analysts, data managers and statisticians at the one end, and the genuine artists and creatives at the other (fiction writers, literary critics, poets) – despise the internet, and those dilettante amateurs like us who populate it. We are self-perpetuating bullshit producers too, but we do it for fun…and for nothing.
We are killing their livelihood, we amateurs. Polls will do on, of course, like journalism will go on. But only those who do so for pleasure – or out of habit – will keep paying for it.
Again: fabulous left-field, deep digging, INQ. In a moment of profound systemic disruption on multiple information fronts, you have to be ‘in the dominant information paradigm, but not of it’ to see it clearly. O ‘Epistemology’ – huzzah, my precious little hillbilly-cod-philosophical neologism, your lexical moment in the sun has come. 🙂
Lots of good points here Jack, e.g. about drill-down targeting, but purely on points 1-7: yeah but, the effects of all of these should be random, so either (1) cancelling each other out if the methodology is halfway OK, or if not (2) producing a dog’s breakfast of results. The results were amazingly consistent, so it’s either herding or some form(s) of un-random misinformation from the public, e.g. (not i.e.) “shy Tory”. Or both. IMO. Not that it matters a hoot. Especially if we are headed for “actual 1:1 scale, real-time, fully comprehensive information pictures of human actuality”. Good luck to all of us with that.
Hi Keith. Yeah, I’m not so sure that’s quite right. About the dog’s brekky/cancelling bit. My point is not that pollsters might not extract consistency, patterns, even swings, as such. It’s how firmly these then are tied to/indicative of material world behaviour. In the end polls are trying to project/predict discrete behaviourial choices from a fraught aggregation of fraught snapshots of fraught individual information responses.
If you ask someone to pick a number between one and ten, or one and a hundred, or whatever…you will get patterns and trends and swings ie group ‘opinions’ too. But can you then base any kind of real world behavioural pattern on that? I think political opinion polls are now similarly moot. Clearly in the last election’s polls that was so. Yes there were trends/‘projected results’. But they were disembodied from material action.
In other words…these polls were not ‘wrong’, as such. What was – is, fatally now – wrong’ is the over-arching premise of ‘opinion polls’. I think that epistemologically they have no more real heft now than, say, chicken entrail readings, tarot cards or astrology. (By the way: opinion polling is a very different thing to psephology, booth and vote analysis, etc…all these are data analytics. Real information work.)
Let me put it this way: the only thing I think any ‘opinion poll’ can now reliably tell us is…how people will answer that opinion poll:
Poll Q: who will you vote for, A, B. Or C?
Results: 50% A, 30% B, 20% C
Material world conclusions:
1. when asked ‘who will you vote for’, 50% will likely answer A, 30% B, 20% C.
2. Likely election result: not known.
This is the reverse/companion version of no-one believing journalism any more. It’s part of the near-complete severance now of human abstraction and human material reality. The mass meeja/mass information class’s most corrosive and yet most-unnoticed legacy. In plain sight, over the last hundred years, radio, TV, mass printing&circulation, for a while the Internet…have slowly turned us schizophrenic. Undermining, bypassing, ameliorating the information hegemony of ‘journalism’ and ‘polling’ – and other enabling modes of info-bipolarity (especially over-monetised information generally, and mediocre thinkers/writers over-amplified by technology) – will redeem our sanity as a sentient species.
The internet is the information tool for doing that, and places like Crikey – and INQ (and Wikileaks etc) – the information forum…
Sorry Keith, I do blah on. My over obsessive hobby horse of the last two decades…
Other components of the information ‘disruption’ currently underway are any non-standard exponents of existing forms. ie meeja that is ‘in the information paradigm, but not of it’:
Newspapers: The Saturday Paper…same familiar physical form but not rendered invisible/moot by the paralysing strait jacket of daily production.
Telly: (limited) Q&A’s recent embrace of peoples panellists…so long as those panellists don’t get duchessed/seduced by TV’s operating imperatives
Radio: being subverted constantly by podcasts.
Anything that ‘f**ks up’ the form but with a constructive, demonstrable improved option. The ‘Journalism’ death spiral is accelerated by the fact that the whining armies of over-paid content rent-boys/broadcast sex dolls foddering to its relentless, pleasureless appetite absolutely refuse to re-invent or even modify their own manifestly dying forms, instead railing arrogantly at the rest of the information world for moving onto more fit-for-purpose epistemologies.
If you want to know how many tonnes of thermal coal Australia exports…only a stupid person would spend three bucks on half a kilo of killed tree, 70% of it printed with stuff you don’t even look at, much less read.
You go here:
https://www.industry.gov.au/data-and-publications/resources-and-energy-quarterly-all
https://www.climatecouncil.org.au/resource/reports/?gclid=EAIaIQobChMI7p-gvcWA5AIVBayWCh2BmQ7AEAAYASAAEgLmxvD_BwE
https://search.abs.gov.au/s/search.html?clicked_fluster=black+coal+exports&cluster0=Coal+exports+&form=simple&origin=-41.176895%2C146.3515&profile=_default&query=%60black+coal+exports%60&collection=abs
Even here:
https://en.m.wikipedia.org/wiki/Coal_in_Australia
All are infinitely better sources of accurate, exhaustive, professionally and vocationally managed information than ‘Journalism’ will again ever be able to competitively offer. Instant, free, multi source cross-checkable, automously husbandable and often interactive. A proper inclusive and permeable, interrogatable human conversation, in other words. Not a one way act of information middle-man ticket clipping.
Thanks, Internet!
Still, the dying old dinosaurs of the ‘mainstream meeja’ screech and bleat and sook and stamp their little footsies in full page open letters ordering us to pay attention to them or our democracy will die and ‘facts’ and ‘truth’ will wither away 4evs!!!!
They just don’t get it. ‘Journalism’ is dying. Hooray. ‘Opinion polling’, a particularly info-wounding epistemological virus it hosts, with die with it. Hooray.
Well Jack ,if blahing on is what you do you do it in an intelligent and informative manner !
I’m not alone in welcoming your input and while I’m undecided about this new comment format I will take advantage of the opportunity to give your “blah” an uptick 🙂
Seconded Mac Tez. And I might press some +s myself.
Of course the other possibility is that the polls were right all along, but on the day of the election there was some sort of accidental miscount.
There is a heavy movement now in areas of depth psychology, exploited by organisations that utilise advanced computerisation for manipulation. Many polls internationally now seem to swing unexpectedly to conservative reserve positions, out of open detection. The unsubtle campaign to bomb, blast and exploit leaves the polling booth decision in doubt. Murdoch knows this as it is part of advertising success. Vance Packard hinted at it long ago. Media bombardment confuses and can deliver a boomerang on polling day.
I don’t think “shy Tory” can be so easily dismissed. If it isn’t herding, and the methods were as diverse as reported, it has to be some form of systematic misinformation. Of course you would have to explain why it operates in some elections but not in others.
Perhaps sometimes, but not always, the nature of the contest and protagonists makes some voters ashamed to admit a vote for the less altruistic option, or to being a class traitor. At other elections, it might be less acceptable to appear to be a bleeding heart. (There is no reason why the phenomenon need always be “Tory” in leaning.) Perhaps there are elections where people find it more difficult to bring to consciousness and articulate the reasons for their vote. In polling, many such people would say “undecided”.
Responding to these polls are NOT compulsory. Since the NBN many are simply giving up their landlines. I for one NEVER respond to polling requests over the phone – including mobile.
After the last Census debacle I am now very reticent at being fully forthcoming on the ‘compulsory’ one – particularly on giving out personally identifiable information.
Anyone using MyGov would know that the amount of information Government is centralizing on individuals is creating an Australia Card by default. I am pretty sure that many Government bodies have an increasingly better handle on electorate profiles than most of us would be comfortable with and would be the envy of the private pollsters.