Privacy advocates raise concerns over Toowoomba council’s iOmniscient surveillance software that they say is ‘invasive’ and linked to racial profiling.
A behavioural surveillance scheme being trialled by a Queensland council in an attempt to anticipate antisocial and illegal acts has prompted concern, with civil liberties advocates saying the technology is “straight out of 1984” and has been linked to racial profiling.
The Queensland privacy commissioner, Philip Green, confirmed he had not been consulted by Toowoomba regional council over its trial of “privacy-invasive” behavioural recognition software with CCTV cameras, which has been linked to racial profiling in the United States.
Toowoomba’s foray into behavioural surveillance technology was also criticised by civil libertarians who warned that people deemed “abnormal” by an algorithm faced being harassed by authorities.
On Monday the council rolled out a month-long trial of “camera analytics software”, which purportedly links with CCTV networks to identify safety risks posed by members of the public.
The technology, sold by a company called iOmniscient, includes facial and number-plate recognition technology and touts an ability to anticipate antisocial behaviour in crowds, such as vandalism and other crimes.
Green said the council had not consulted his office about the rollout of technology despite it raising serious privacy concerns – the second time in a month a Queensland council has done so.
“Frankly I don’t know what [the scheme] is, I have no visibility on it, they haven’t consulted with us to my knowledge and I’d like to look at it,” Green told Guardian Australia.
He said “well-intended” councils were “trying to explore new technology” but in this case the move provoked “some serious debates about profiling people”.
“That’s where I think it can get scary because facial recognition’s not that accurate, mood recognition as Facebook’s trying to run out or whatever Toowoomba is trying to do with behavioural pattern recognition – all those algorithms have failures,” he said.
“There’s a wider debate that’s beyond privacy, around the adequacy of the decision-making process based on it and that’s a wider thing for artificial intelligence generally.
“If they’re using a privacy-invasive technology in the first place to base their algorithms on, there’s a few concerns across the board.”
Michael Cope, the president of the Queensland Council for Civil Liberties, said the technology was “straight out of 1984” and had been linked in the US with a tendency to over-select racial minorities.
“The algorithms which underlie this technology have at their heart arbitrary concepts of what is normal,” Cope said. “People are selected for attention by authorities on the basis of their supposedly abnormal behaviour or appearance.
“How would you feel if you are innocently minding your own business in the mall and you are approached by the police in front of everyone because a machine thinks you look odd?”
iOmniscient’s Sydney office could not be reached for comment.
But the company website details a product called IQ-120 which “has the ability to detect behaviour deemed suspicious such as loitering or running, based on the speed and pattern of their movement”.
Staff in CCTV monitoring rooms can “more closely monitor people whose behaviour seems suspicious, potentially averting crises before they happen”, it says.
The software can be customised to identify behaviour a client deems “suspicious for their situation”, it says.
It gives the example of tracking potential thieves in car parks by configuring software to “recognise that a normal person would walk directly to their car while generating an alarm for any person that is seen to walk form car to car as they decide which one to steal”.
It could also detect “aberrant behaviour” such as when “a suicidal person may loiter in one place as they build up courage to blow themselves up, or before they jump off a train platform”.
Cope called on Toowoomba council to reveal what data from its records was being “fed in to this software and what data is being fed back into the databases”, adding: “We object to the creation of vast databases.”
The use of behavioural recognition in US airports had resulted in “many innocent people being harassed by security staff”, he said.
“The problem is that when a machine makes the assessment, the pressure is on for the operator to take action lest they be accused of negligence,” he said. “The result is that the number of innocent people being accosted by the authorities will go up.”
Green said it was preferable if councils “could talk to us about it before they roll out” technology that clearly infringed on people’s privacy. This would allow a “privacy-impact analysis [because] that’s going to be the standard internationally” and an assessment of whether it was “in proportion” with its purpose.
It was important that the public was involved in a debate about the “proportionality” of such schemes, he said.
The Gold Coast city council has consulted Green’s office on a trial of facial recognition surveillance in the lead-up to the 2018 Commonwealth Games. Green said even with the security risks around such an event, given the high-level presence of law enforcement, he personally wondered “whether councils should be getting into that sort of thing”.
The commissioner is awaiting a detailed response from Moreton Bay regional council to his queries about its rollout of audio recordings with its CCTV camera network last month. He previously said he was concerned the scheme might breach the state’s Information Privacy Act, as well as criminal law in the Invasion of Privacy Act.
Cope said Queensland councils with camera surveillance had “consistently behaved as if the Information Privacy Act does not apply to them”.
On Tuesday a Toowoomba councillor, Geoff McDonald, said the trial would give other councils a chance to see potential benefits for crowd management, locating missing objects and helping to find missing people.
He said there were “potential community safety benefits through automatic or early identification of antisocial behaviour and potential safety risks”, according to the Sunshine Coast Daily.