Dan Andrews’ announcement of a slow, cautious return to normal left many Victorians frustrated. But the premier says he has no choice — modelling done for the health department by the University of Melbourne showed that if the state eased restrictions too soon, Victoria would have a 64% chance of being locked down again by Christmas.
“You can’t argue with this sort of data, you can’t argue with science, you can’t do anything but follow the best health advice, otherwise … we will just be beginning to lose control again of this virus,” Andrews said on Sunday.
But despite Andrews’ claims that the numbers don’t lie, there’s been plenty of criticism of both the modelling, and the lack of transparency around it.
What do we know about the model?
The University of Melbourne team used what’s called an “agent-based” model. Agents, or people, move around like pieces on a chessboard. When a person enters another person’s square, they become infected with COVID-19. As restrictions are added or relaxed, the way they move around the chessboard changes.
The researchers used a supercomputer to run 1000 simulations of the chessboard, with variables put in place — factors like compliance with mask-wearing, observance of social distancing,
What we don’t know
The problem with Victoria’s reliance on a model to justify continuing restrictions is there’s a lot about the model we just don’t know, says Deakin University epidemiology chair Catherine Bennett.
“The model depends on the parameters you put in it, and this model appears to be built from info we knew earlier on from various settings around the world,” Bennett told Crikey.
Without seeing those parameters, we can’t get a firm picture of how good the modelling really is. Instead, there’s not been a huge amount of transparency around it, Bennett says.
“We don’t know what questions were asked of the modellers by the government, or what they tested,” she siad. “We don’t know if they’ve updated the parameters. We don’t know what the scenario for opening up was.”
Andrews’ big claim — that there’s a 64% chance of returning to lockdown if restrictions are eased to soon — is a case in point. The modelling doesn’t tell us whether that’s in a scenario where there are no restrictions at all, or whether even if some restrictions are in place a return to lockdown is likely.
The modelling also doesn’t account for different rates of transmission in certain areas, like aged-care settings and among health care workers.
Speaking to ABC Radio National Breakfast yesterday, Doherty Institute director of epidemiology Jodie McVernon also said the modelling provided “scant” detail.
“There was very little detail of the modelling that was presented yesterday and I think it was probably disappointing that we didn’t hear more of a description of what the locations of infections were,” McVernon said.
Does it matter?
Modelling isn’t a magic bullet to save Victoria but should just be one element underpinning a policy decision, according to Flinders University professor of medicine Nikolai Petrovsky.
“A model is useful, but should never be what determines your policy. Real world evidence, what happens in other countries, all should be considered,” he said.
Petrovsky said the toughest question facing Victoria right now wasn’t about the strength of the modelling, which clearly showed how distancing could lower case numbers, but whether a longer lockdown was the best way to deal with the current situation.
UNSW epidemiology Professor Mary-Louise McLaws says while the Melbourne University modelling seemed thorough, what mattered was that it had helped guide the Victorian government towards an approach she says is the right one.
“Modelling is a helpful element, that experts use to make a decision — a model doesn’t make the decision, it just helps,” McLaws said
“Hearing that they’ve done over 1000 iterations with several scenarios, knowing the limitations of models, I think they’ve done a very thorough job.”
With case numbers over the last two weeks falling but still high, McLaws said the critical issue was maintaining the cautious, conservative easing of restrictions necessary to beat the second wave.
“The critics have failed to understand that it’s the decision that matters.”
Is it enough to trust the modelling? Let us know your thoughts by writing to letters@crikey.com.au. Please include your full name to be considered for publication in Crikey’s Your Say section.
I know – let’s find a panel of fifty different experts, each with their own opinions of the best thing to do, and let’s do nothing about the virus for six months while they sort out who’s right and who’s wrong. And let’s wait until the new case count is back up in the thousands per day and we’re back into Stage 4 lockdown for the rest of the decade. THEN, let’s all have a great time kicking the heck out of the Andrews government for not being decisive and not acting quickly enough.
It’s always easy for those safely on the back lines of any trench warfare to lob criticisms in the direction of those on the front line facing the enemy.
I don’t want to kick the heck out of Andrews, I voted for him.
I just don’t want him rewarded for kicking the stuffing out of Victorians.
Using the war metaphor you use, it happened on his watch and he is responsible.
You really do need to get commentators who have some capacity to understand what it is they are commenting on. Assembling a series of uncontextualised quotes is not analysis. It is not even an anecdote.
The implication that only the output from one model run is used to make decisions in a mechanistic way is churlish and unhelpful. That is quite obviously not what is happening.
Saying that decision making should take into account real world experiences implies, very strongly, that this is not being done, when in fact it obviously is being done. Was the person quoted as saying these things simply commenting on the general process of decision making around a model? Or were they, as presented here, specifically criticising the decision making process in this case?
To wonder out loud whether parameters are being updated is both snide and churlish. Of course they are! To imply that one of the best epidemiological modelling outfits in the world with one of the best models and, more importantly, one of the best processes for parameterising that model has somehow gone to sleep on the job, is professional jealousy gone mad.
Fortunately, the sillies are dealt with by quoting someone who appears to understand how models are used. Including this one.
“Trust the science”, “look closely at the modelling”, “respect the data”. These are all superficially incontestable statements, especially when you consider the counter factual (“make it up as you go along”, “spin a wheel”, “ask your Facebook friends what they think”).
But science is not, and has never been, determinative. Science is based on probability; the more complete the data set, the more rigorous the analysis, the higher the probability of certain events occurring. And the concept of a complete data set is somewhere between abstract and unachievable.
The modelling used by the Victorian Government appears (from what we’ve been able to glean) about the spread of the virus and, based on assumptions of health impact (which presumably incorporates demographic factors … ), the consequence for the health system. There are other models that could be thrown into the mix – the likely impact of restrictions on mental and physical health (which itself has an impact the health system), economic consequences etc etc. Whether such models would or should change the current policy direction isn’t clear, but simply asking the question isn’t heresy. I’m sick of the binary ‘You either support lockdowns (#standbydan) or you don’t care about human life (#dictatordan)’ debate we’ve fallen headfirst into. Whatever happened to nuanced, constructive debate?
Yes, cleave to science and data as the basis for decision-making, but let’s not fool ourselves that ‘science’ is about a definitive and incontestable answer. Science should be about questioning.
I can only admire Andrews for standing his ground and maintaining his confidence in his own judgment and the counsel of his experts, in the face of so much sniping from the peanut gallery.
The snide behaviour of Morrison, Frydenberg, and Hunt is particuarly galling as Victoria does the hardcyards with genuinely talented leadership.
Dr Swan said it all this morning: NSW is not the gold standard, it is just lucky.
Vic Chief Medical Officer Brett Sutton said today that he was never for the curfews and that it was a “Captains call” by Dan. So Dan doesn’t need evidence, he just needs power. He directly went against the medical advice. What else is he doing that doesn’t need to be done?
PS I am using Vic Chief Medical Officer Brett Suttons words, not paraphrasing.
Everyone is an expert, but no expert is as expert as an expert who wasn’t consulted and missed their moment of fame.
I remember being told way back when; Ex = past tense, spurt = a drip under pressure. therefore an “expert”, is a has been drip under pressure.