When Algorithmic Bias Turns Deadly

Machine-learning models are notoriously susceptible to algorithmic bias, particularly when it comes to people of color. Just a few years back, software used by the US criminal justice system was shown to disproportionately suggest black people were more likely to commit crimes. Then there was the time that Google's image-recognition system identified African Americans as gorillas. But now this could be an even bigger problem than originally thought. A new report suggests that the object-detection models used by self-driving cars are significantly less capable of identifying dark-skinned pedestrians than those with lighter complexions. While the study is certainly no smoking gun for opponents of this controversial technology – it has yet to be peer reviewed, and it used an approximated model and data set rather than one currently used by self-driving vehicle manufacturers – it still brings to light an important possibility.