As the profound flaws and poor track record of facial recognition become apparent, Australian lawmakers, especially at the state level, look set to be left behind in an international move to block a technology that is already wrecking lives.
Overnight, Boston banned the technology, following Oakland and San Francisco which banned it last year. Boston’s ban had the support of the Boston police, who declared the technology unreliable. States such as California, New Hampshire, and Oregon have more limited bans and a number of other states have restrictions on its use.
A late 2019 paper by the US National Institute of Standards and Technology examined both one-to-one matching (matching a facial image with another facial image) and one-to-many (matching a facial image to a database of images). One-to-one facial recognition had a vastly higher rate of false positives for Asian, African-American and Native American faces, while one-to-many systems had a significantly high rate of false positives for African-American women.
Funnily enough, systems designed outside the US delivered fewer false positives for non-white faces.
The study confirmed earlier studies showing facial recognition systems, relying heavily on white, and male, facial features, often fail with gender and racial diversity. Unsurprisingly, a Black man was at the centre of the first case of US police arresting an individual after a failure by facial recognition systems.
Earlier this month, IBM said it would no longer sell facial recognition tech, saying “vendors and users of Al systems have a shared responsibility to ensure that Al is tested for bias, particularity when used in law enforcement, and that such bias testing is audited and reported.” In January, Google supported EU consideration of a temporary ban, and later launched legal action against the company behind Clearview, which scrapes social media and the internet for facial data and provides it to law enforcement.
Facial recognition is a large and rapidly growing industry. One estimate suggested it was worth US$3.4 billion globally already and is expected to grow at over 14% a year in coming years.
In Australia, police have been given a free hand to use systems demonstrably biased against non-white individuals, despite the Human Rights Commission calling for moratorium on the technology. Scandal-plagued NSW minister David Elliott even boasted about the use by NSW police of it. Queensland has begun using facial recognition systems at sports stadiums while uploading all Queensland drivers licence photos to a national facial recognition database.
The only political pushback came last year from federal parliament’s joint intelligence and security committee, which unusually outright rejected an attempt by Peter Dutton and Mike Pezzullo to establish the basis for a national facial recognition system without privacy safeguards, oversight, accountability or recognition of the deep biases of facial recognition systems.
However, this only addresses the wild overreach by national security bureaucrats at the federal level; the primary problem is at the state and territory level, where police forces have been able to exploit surveillance technologies without check.
The federal Privacy Commissioner earlier this year launched an inquiry into the possible use of Clearview by Australian police forces. But with persistent evidence of racial bias in Australian police forces and the vast overrepresentation of Indigenous Australians in the criminal justice system, the potential for errors and outright misuse in Australia is rapidly growing without check.
Correction: This article originally referred to the NIST paper as “peer-reviewed”; the NIST has contacted Crikey to advise it was not peer-reviewed as per an academic paper. Crikey has removed the phrase.
OMG, racist computers ! Where will it end ?
Ask anyone who knows anything formal about photography and they’ll tell you about contrast range and how the human eye is much better at it than any film or photosensor.
Try looking at white and dark faces out on a sunny day with the sun behind them and the difference in discerning facial feature is immediately apparent. I’m sure all this is already well known to those in the business.
The broader issue of the ethics, reliability and suitability of such practices is important though. We’ll get the usual weasel words about catching terrorists and child abusers but it will soon morph into tracking people who haven’t paid toll road fines and so forth.
This is the government that instituted Robodebt.
Robodebt, Robocop, Roboratbag – very consistent, this lot.
Always sure to go low, lower & lowest.
Agree
Ban it immediately, stop funding anyone doing research on it and start shaming those who continue to do so.
Put the onus on them to justify why this repulsive technology should ever be allowed to continue.
Because, Laura Norder.
I’ll wager ‘Clearview’ couldn’t distinguish Spud Dutton from a line-up of some of the worst dictators in history.
Where can I buy a Spud mask?
There’s an old one at life size cutouts (he has hair). I think an update is due. Though personally I think the rubber style ones would be better.
… A world of disobedient Spuds …. Sontarans? Straight out of Dr Who?
He always looks as if he’s wearing a stocking mask so it should be no problem copying.
or a potato…
Facial recognition will be used and it will be abused in Australia. It is very probably already happening.
Add it to warrantless access to mandatory meta data retention, warrantless tracking devices, very low thresholds for other hyper surveillance methods including state based spyware and encryption breaking technologies and all tied into a mega database and computer system and you have something approaching Orwell’s “boot stamping on a human face forever”.
We know the government won’t tolerate whistle blowers and will punish them in anyway they can. The system they are creating can be used not just to investigate whistle blowers and dissidents but to punish them in an extra judicial manner, even when the law can’t.
For example, even though not from Australia this story shows the kind of things that can and do happen when the state doesn’t like someone.
http://web.archive.org/web/20181130165344/https://policestalking.wordpress.com/2012/07/15/an-introduction-to-police-stalking/
The slope we are on is so slippery it’s greased, and we are already half way down. Almost no one seems to care though or even understand the consequences of what is happening.
Thanks for this comment and the link, Lovert. I’m pretty sure this is what was done to me about 20 years ago, even though I can only guess why this trash targeted me. Don’t even want to know because as the people in the article say, there’s nothing you can do about it, except move.
I’m sorry to hear that Oldie. They can break people with these methods. Some people do know why they are targeted. It could be for example they are involved in drug law reform efforts.
Unfortunately as the article mentions relocating to another geographic area will not help, especially with the way technology is being used in today’s world.