Australia's Royal Commission into the Robodebt Scheme has published its findings. Various unnamed individuals are referred for potential civil or criminal investigation, but its publication is a timely reminder of the potential dangers presented by automated decision-making systems, and how the best way to mitigate their risks is by instilling a strong culture of ethics and systems for accountability in our institutions. The so-called Robodebt scheme was touted to save billions of dollars by using automation and algorithms to identify welfare fraud and overpayments. But in the end, it serves as a salient lesson in the dangers of replacing human oversight and judgement with automated decision-making. It reminds us that the basic method was not merely flawed but illegal; it was premised on the false belief of treating welfare recipients as cheats (rather than as society's most vulnerable); and it lacked both transparency and oversight.

Automation in public administration is inevitable and can bring great benefits. The broadly accepted law of robotics is that a robot may not injure a human being. In an attempt to reduce welfare costs in 2016, the commonwealth government engaged in an unlawful debt recovery process. The bureaucratic process was malign and was meant either directly or collaterally to harm and stigmatise welfare recipients. The Online Compliance Intervention – or OCI, but more commonly known as robodebt – used algorithms to average out incomes of welfare recipients by matching ATO income data with social welfare recipients' income as self-reported to Centrelink with Centrelink.

The UK's Department for Work and Pensions (DWP) is accelerating its use of automated systems to handle claims for benefits, in a move some fear will disadvantage welfare recipients. The DWP has hired nearly 1,000 new IT staff in the last 18 months, and increased spending to about £8m (US$10.3m) This new'virtual workforce' is to take over some of the jobs of humans. According to an investigation by The Guardian, which has unearthed further detail on the DWP's plans, one recent tender document requested help to build "systems that… can automatically carry out tasks without human intervention". As well as contracts with a number of multinationals, the department is working with UiPath, a New York-based firm co-founded by Daniel Dines, the world's first "bot billionaire", whose machine learning software is being deployed by the DWP to check benefit claims and detect fraud.