What If the Data Tells You to Be Racist? When Algorithms Explicitly Penalize

Original published in The San Francisco Chronicle (the cover article of Sunday's "Insight" section) What if the data tells you to be racist? Without the right precautions, machine learning -- the technology that drives risk-assessment in law enforcement, as well as hiring and loan decisions -- explicitly penalizes underprivileged groups. Left to its own devices, the algorithm will count a black defendant's race as a strike against them. Yet some data scientists are calling to turn off the safeguards and unleash computerized prejudice, signaling an emerging threat that supersedes the well-known concerns about inadvertent machine bias. Imagine sitting across from a person being evaluated for a job, a loan, or even parole.