Delivering the benefits of artificial intelligence to everyone requires creating systems that work well for everyone. At F8 2019, we highlighted Facebook AI's range of systems and processes currently in production for developing inclusive AI and addressing labeling bias, algorithmic bias, and intervention bias. These efforts help ensure our computer vision (CV) systems work well for all skin tones, for example, and allow our augmented reality effects to serve everyone regardless of facial features, hairstyle, or other factors. But creating fair and unbiased AI systems will require mitigating other forms of potential bias as well, so today, Facebook AI researchers have published the first systematic study that measures the accuracy of object-recognition systems for different communities across the world. Events such as weddings or commonly used household items (dish soap, for instance) can look very different in different places, so CV systems trained with data predominantly from one region may not perform as well when classifying images from somewhere else.

They comb through databases of previously labeled images and look for combinations of visual features that seem to correlate with particular objects. Then, when presented with a new image, they try to determine whether it contains one of the previously identified combinations of features. Even the best object-recognition systems, however, succeed only around 30 or 40 percent of the time -- and their failures can be totally mystifying. Researchers are divided in their explanations: Are the learning algorithms themselves to blame? Or are they being applied to the wrong types of features?