‘Debt’ or alive, you’re coming with me: What ‘RoboCop’ and Robo-Debt can teach us about the move towards AI.
The film ‘RoboCop’ is about a murdered police officer turned into a superhuman machine programmed to kill criminals. Throughout the film, RoboCop has tiny flashbacks to his previous life, and by the end calls himself his living name, Murphy, regaining his humanity. While the fictional world has RoboCop, Australia has ‘RoboDebt’.
Robo-debt, formally known as the Online Compliance Intervention (OCI) system, was introduced in 2016 by the Department of Human Services (a.k.a., Centrelink), to identify and enforce the recovery of welfare fraud: it was the ‘machine’ to catch the veritable ‘bad guys’. Robo-debt used computerised data-matching to compare an individual’s fortnightly Centrelink payment against their averaged fortnightly income based on annual tax returns. Those identified as having earned above a set threshold were sent an automated debt notice. At the height of the scheme, 20,000 notices were sent each week, with the onus placed on the recipient to prove that the debt didn’t exist. Notice recipients, many of whom were vulnerable members of society, with limited financial resources and records, were deemed guilty until they could prove they were innocent. Further, contacting Centrelink to dispute the debt was extremely difficult, with 55 million phone calls met with a busy signal.
Centrelink had used this same method of data matching before robo-debt. However, it was used to identify potential cases of fraud, which were further investigated manually, before a debt was issued (Carney, 2018). Robo-debt removed this step, stripping the machine of its humanity. The complexities of a sensitive, human situation that affected marginalised people, were reduced to an algorithm that compared apples with oranges.
The robo-debt fallout has been enormous. Notice recipients described experiencing heightened stress, anxiety and depressive symptoms. Some reported taking out personal loans to pay the debt, feeling scared and powerless to prove their innocence. Suicides were reported. A class action against the scheme has nearly 10, 000 plaintiffs. A Senate inquiry concluded that robo-debt lacked procedural fairness at every stage, which “disempowered society’s most vulnerable citizens, causing emotional trauma, stress, and shame” (Community Affairs Reference Committee, 2017).
Robo-debt saw our worst fears about AI come to life, a worrying indictment given Australia ranks relatively well in the 2019 Government AI Readiness Index at 11th. However, there’s also a lot we can learn from Robo-debt moving forward. With PWC indicating that AI technologies will add US$15 trillion to the global economy by 2030, many are jumping on the bandwagon. Robo-debt has shown us that we need to consider how the machines can do their job effectively while retaining humanity. We need to make robo-debt like RoboCop, turning Centrelink into the protagonist in this story. Encouragingly, Centrelink has shown progress towards this: the program has been overhauled, with the reliance on income averaging removed. More evidence is now required before a debt notice is issued, and all existing debts have been frozen for review.
In Europe, humanity comes in the form of the General Data Protection Regulation (GDPR): a part of EU law that excludes certain decisions from being fully automated and creates legal rights for those affected by automated decision making. While Australia has guidelines for automated decision-making, we do not have laws to regulate it or protect citizens from any negative ramifications.
According to QUTeX Professor Melinda Edwards, facial recognition systems, deepfake videos, and automated social media bots are early, worrying examples where the power of the technology has outstripped our ability to regulate it.
“With many unintended consequences of AI, we need to understand how we can regulate it and ensure we have protective systems in place that can help us to trust AI”.
Professor Edwards will join Professor Dan Hunter, Executive Dean, QUT Faculty of Law, and international expert in internet law, to explore how the law can respond to the rise of AI systems and the areas where law and ethics that will be challenged by AI.
https://www.qut.edu.au/study/professional-and-executive-education/courses/real-world-futures
Sources
https://www.oxfordinsights.com/ai-readiness2019
https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificial-intelligence-study.html
https://www.oxfordinsights.com/ai-readiness2019
Carney, T. (2018). The new digital future for welfare: Debts without legal proofs or moral authority. UNSWLJ F., 1.
