Of the extensive list of reasons that can be offered as to why the apparent defeat of the Liberal National Party in Queensland came as such a shock, none looms larger than the fact that the late opinion polls told us it wasn’t going to happen.

As political obsessive-compulsives glued themselves to the internet throughout Friday evening, when they could so very easily have just waited a day, three successive pollsters rang up essentially the same result: 41% to 37% in favour of the LNP on the primary vote, translating into a 52-48 lead on two-party preferred.

As was explained here yesterday, the first part of this equation was more than accurate enough to satisfy a fair-minded observer of the polling game. However, it was the two-party headline numbers that ricocheted around the social and news media throughout the following day, and those numbers were plainly wrong.

For pollsters in Australia, how to determine the two-party preferred vote is an increasingly vexed question.

With the combined minor party and independent vote having increased progressively (if rather unevenly) from a little over 10% in the 1980s to a little over 20% today, a set of primary vote numbers increasingly fails to answer the question on everybody’s lips.

However, translating accurate primary vote numbers into accurate two-party preferred numbers is often easier said than done.

For many, voting is conceived as a simple question of choosing which party to support, no different from how it would be in the vast majority of democracies that lack Australia’s system of preferential voting.

But while the question of how the rest of the ballot paper should be dealt with may be an afterthought in the mind of the voter, it’s no small matter so far as the election result is concerned, since a vote received as a preference counts every bit as much in the final analysis as a primary vote.

Very often the matter will be determined by how-to-vote cards, which are available to the voter at the polling booth, but not when responding to an opinion poll.

A further difficulty is the tiny sub-samples of voters for given minor parties captured by a typical opinion poll.

If a poll has 1000 respondents, as is generally case when the expensive but reliable method of live interview phone polling is used, a Greens primary vote of 10% allows for only 100 respondents from which to project the final destination of over a million votes.

The laws of random sampling dictate that such a result will bounce around from one poll to the next in a range approaching 20% — very often making a full percentage point of difference to the two-party total, when in fact all that is being picked up is statistical noise.

In the face of such issues, the industry standard for the past decade has been to assume that minor parties’ preferences will divide between the major parties as they did at the previous election.

So long as the minor party landscape remains much the same from one election to the next, there is no question that the imperfections of this approach are less than those of the more intuitively appealing alternative of asking respondents what they will do.

However, that qualification is an important one. Previous election preferences would have offered little guide in 1998, for example, when Pauline Hanson’s One Nation came from nowhere to poll over 8.4% of the national primary vote.

The strongest challenge to the assumption of minor party stability in recent times came in 2013, when the newly established Palmer United Party polled 5.5%. And sure enough, that election brought a stronger flow of preferences to Labor than in 2010, although a 4% increase in Labor’s share of Greens preferences was also a factor.

This caused Labor’s two-party vote to come in a full percentage point higher than 2010 election preferences would have suggested, partly explaining the party’s relief on election night that things hadn’t turned out quite as bad as feared.

Even so, the previous-election approach still proved more reliable than respondent-allocated preferences would have, at least if the numbers published in the lead-up to the election by Nielsen were anything to go by. The trend of Nielsen’s respondent-allocated results had Labor’s overall preference share around 8% too high, causing an overshoot of 1.5% if applied to the primary vote result of the election.

Then came Saturday’s Queensland election, a result remarkable in every way. Owing to the dramatically different behaviour of preferences when compared with 2012, the headline results of 52-48 to the LNP from Newspoll, Galaxy and ReachTEL would have been much nearer the mark if they had come in the other way round.

It’s surely no coincidence that this failure occurred under optional preferential voting, for the decision to opt out of allocating preferences altogether allows for much greater volatility than the question of which major party a particular cohort of voters dislikes the least.

The exhaustion rate of Greens preferences shot up by 10% at the Queensland state election of 2001, and sank a good deal more heavily than that on Saturday. But you will never go too far wrong with the rule of thumb that Greens supporters who number every box, whether willingly at a Queensland state election or by compulsion at a federal one, will favour Labor over the Coalition by a ratio of about four to one.

With the other state that uses optional preferential voting set to go to the polls in less than two months, this would seem to present pollsters with a pressing question.

Faced by the looming test in New South Wales, Nick Adams of ReachTEL says the pollster is “actively investigating providing respondent-allocated preferences in addition to previous-election preference allocation methods”.

This reflects the practice of Roy Morgan and Ipsos, whose respondent-allocated results have consistently been the more favourable for Labor.

However, this approach is criticised by David Briggs, managing director of Galaxy Research, who describes it as “an attempt to have two bites at the cherry in the hope that one is correct”.

Briggs describes the rate of exhausted preferences as “just another variable that we need to factor in”, and says that beyond “fine tuning”, he is “not expecting a significant change to our approach”.

In light of Saturday’s experience, that might be thought a bold call. For the opinion poll consumer, it might seem advisable to treat two-party results ahead of the New South Wales election with due caution, while remaining broadly confident in the primary vote numbers.