Dutch court rules AI benefits fraud detection system violates EU human rights ZDNet

A Dutch court has demanded that an algorithm-based system used by the government to identify and track down potential housing and benefit cheats is dropped with immediate effect. As reported by DutchNews, on Wednesday, the District Court of The Hague ruled that the system conflicts with EU human rights and privacy protections. Dubbed System Risk Indication (SyRI), the automatic, machine-learning (ML) tool was used by local Dutch authorities to draw up profiles and lists of individuals suspected of being at high risk of conducting benefits fraud. According to the publication, SyRI creates risk profiles from individuals that committed social security fraud in the past and then scans for "similar" citizen profiles, creating leads for potential investigations into others that may also be committing fraud, or be of a high risk of doing so in the future. SyRI's pooling of citizen data, otherwise kept in separate silos, gave authorities wide-ranging powers and "has been exclusively targeted at neighborhoods with mostly low-income and minority residents," according to UN human rights and poverty rapporteur Philip Alston.