The federal election result appears to be the tipping point in public confidence in political polls. Of the hundreds of recently published polls and exit polls, can anyone find one predicting a win for Scott Morrison? Can political polling now be trusted to accurately predict an election result?
The answer could be that we are now in an era of “fake polls”. And more disturbing, those fake polls are the basis for an industry of academics and journalists who rely upon them to provide analysis. If the polls are fake, the news and commentary can only be wrong.
There was a time we had confidence in polling results. Here are some of the issues which have taken political polling to an existential crisis.
Representative sampling
The magic of polling was introduced by Gallup in 1936. Polling would be accurate if it captured, in microcosm, a representative sample of the public. Butchers, bakers, candlestick makers were assessed in microcosm and their views extrapolated to the broader national public. For some decades this worked brilliantly.
But things have changed very rapidly.
Think of a “representative baker” now. Is it the hipster in Newtown or Northcote baking artisanal sourdough, or the employee of a Coles bakery? And what is the baker thinking? What are his points of engagement with the world, the things that help shape his attitudes?
A few decades ago we could predict with significant accuracy the stimuli he would be exposed to. Today, is it Netflix bingeing, social media, free-to-air TV, or the infinite spectrum of the internet and immersion in algorithmic designed filter bubbles?
How do we sample his mindset today? How do we extrapolate this mindset? How many are there like him?
The collapse in response rates
If there is a problem in defining representative samples, there is a greater problem in simply getting people to participate in a poll.
Participation rates have collapsed. Once a public duty, there now appears more a type of person who will respond to a poll, and as importantly a type who will not. We have shifted from most people willing to participate in a poll to overwhelmingly most people not wanting to participate.
Not wanting to participle in something has meaning. What are the views of the type of person who does not participate in a poll? We simply don’t know.
Two-party preferred
The key paradigm of major public opinion polls is flawed. The polls measure a supposed swing between the major parties. This is represented in the two-party preferred result — 2PP. But the 2PP paradigm is a stark distortion of public voting behaviour.
If there is a swing in modern Australian politics, it is a swing against the major parties, rather than between them.
The collapse in major party identity is a key ingredient in the current inability of published polls to provide accuracy. For example, over the last 25 years both the ALP and Coalition have slowly reduced their primary votes. The ALP has lost over 25% of its primary vote in 25 years.
The very voters who have rejected the two-party option are now being squeezed and disfigured to fit the 2PP poll paradigm.
Traditional voters who have always voted the same way provide a great service to the current polls. They are the reason we do not have high volatility of results in political polling, and we do not get wild poll swings of 10% or 20% in error. The traditional voters anchor polls. For example, if 75% of voters don’t change their vote, any volatility is quarantined to the remaining 25%. So it is easy to see why even failing predictions are reasonably close to the final voting result A 2% error across the whole sample looks accurate, but when applied to the volatile area, represents an error rate of 8%.
This has masked the emergent errors in polling. But the voter anchor is rusting away; the slow erosion of party identity is adding to the inability to rely on predictive political opinion polls.
There was no “bang” or key moment to mark the point that political polling failed. It is the recent and accretive impact of many factors. There is really no one to blame, and it is not quarantined to Australia, but we must recognise there is a problem with our supposed ability to predict election outcomes. The magic has gone.
How much faith do you have in the polls? Send your comments to boss@crikey.com.au.
Could be the Russians?
What useful purpose do public polls play, other than the inclination to a herd mentality for the dim of wit?
To know if a party’s message is popular/cutting through.
It isn’t very good for our politics if politicians waste time with policies nobody wants because they don’t realise nobody wants them.
The 3% difference from the pollsters vs result nation wide is hard enough to stomach, but possibly within random chance plus less reliable access to random sampling.
The complete failure of QLD polling, with expected 50/50 turning out to be 60/40 seems only explainable if no one actually did polling.
With the exit polls apparently showing the same results as the pollsters, were the pollsters actually trying or just deciding what the numbers should be without the costs of actually asking anyone? This should have been easier to get a correctly weighted sample.
I can understand betting markets ‘herding’ towards published opinion polls, but we all should have been more critical of 50 polls in a row and so many that were so close to each other.
https://marktheballot.blogspot.com/2019/05/why-i-am-troubled-by-polls.html – published the Wednesday before the election appears to be prescient.
“… I remain troubled. A systemic problem with the polls, depending on what it is, may point to a heightened possibility of an unexpected election result…”
Keith,
The polls did not take into account that 3 percent, so the 3 percent is worthless.
They all said Bill Shorten would win. That is not a plus or minus 3 percent. That is a guarantee, put your house on it Bill Shorten and Labor was a guaranteed winner. So that is how ridiculous the pollsters are arguing that they were within the 3 percent. That is incorrect. They were 100 percent out in that Bill Shorten and Labor did not win.
I don’t think polling will be trusted for a long time. Misinterpretation of the Trump “miss” (the polling was right about the popular vote and even hinted at the states where Trump might be ahead to give him a path to victory despite losing the popular vote), the actual miss on Brexit, the Vic state election miss (which was I think even bigger in raw terms than this one, but since the miss didn’t get the actual result wrong it didn’t quite tip over the polling apple cart) and now this miss which got the national wrong, the states wrong, everything wrong in 15 consecutive polls across the 4 major polling companies in Australia….
It does seem most likely that the pollsters in Australia have got their sampling wrong and/or the people who don’t answer polls tilt it (not necessarily pro-conservative, since the Vic miss was in the other direction, but tilt it nonetheless). It will take a while not only for them to work out the error and correct it, but then to get enough correct results to persuade people to believe polls again.
Professional opinion polling is done for. If you’re in the industry, look for a new job, if you’re nearing retirement hang around for the redundancy.
For polling to work it requires a representative sample. If you don’t get that all the maths doesn’t work, and society is now far too atomized to get a representative sample.
It will get some correct in future. A broken clock is right twice a day, but this problem is insurmountable for polling.
But don’t expect them to volunteer that, even if they know it.
I’m selecting you DB because you are one of the more sophisticated readers.
> and society is now far too atomized to get a representative sample
Just what crystal ball were you gaping into when that gem of an inspiration materialised DB?
As I have pointed out elsewhere (on these pages of late) random (or representative sampling) is an everyday event using real tools and methods with real money. Moreover, sampling is not a monolithic activity; there are a myriad of forms of sampling and each has strengths and weaknesses. To complicate matters, the responses are nowadays, typically, highly correlated : hence the scew-ups with Brexit, Trump and (I guess) the weekend.
Bastardising the phenomena, for the sake of illustration, if response G is related to response B or E or W or all three (or 300) then the effect of the responses become difficult (sometimes impossible) to manage and quantify in isolation. See my posts here and elsewhere (of today) on related topics.
As with the law, one gets the data one pays for. If the firm or business is happy with the material that has already been through the horse it generally comes cheaper. Farce bonk (FB) is not in business for nothing.
As an aside, Assange gave the stuff away and is regarded as a crim. Zuckerberg, employing fair means or foul, sells the stuff and is regarded as a business luminary. Different huh?