So far, the way we've tried to answer that question is through benchmarks. These give models a fixed set of questions to answer and grade them on how many they get right. But just like exams like the SAT (an admissions test used by many US colleges), these benchmarks don't always reflect deeper abilities. Lately it feels as if a new AI model drops every week, and every time a company launches one, it comes with fresh scores showing it beating the capabilities of predecessors. On paper, everything appears to be getting better all the time.

It's no secret that artificial intelligence, algorithms, and big data have a problem with gender and racial bias. These systems can be biased based on who builds them, how they're developed, and how they're ultimately used. Trying to solve the problem is a community of Black data scientists, researchers, and organizations. This article highlights the Black women amongst their ranks, who are exposing algorithmic biases, empowering communities of color with data, and arguing for more diverse representation. Joy Buolamwini is a Ghanaian-American computer scientist based at MIT Media Lab.

Cassie Kozyrkov has served in various technical roles at Google over the past five years, but she now holds the somewhat curious position of "chief decision scientist." Decision science sits at the intersection of data and behavioral science and involves statistics, machine learning, psychology, economics, and more. In effect, this means Kozyrkov helps Google push a positive AI agenda -- or, at the very least, convince people that AI isn't as bad as the headlines claim. "Robots are stealing our jobs," "AI is humanity's greatest existential threat," and similar proclamations have abounded for a while, but over the past few years such fears have become more pronounced. Conversational AI assistants now live in our homes, cars and trucks are pretty much able to drive themselves, machines can beat humans at computer games, and even the creative arts are not immune to the AI onslaught. On the flip side, we're also told that boring and repetitive jobs could become a thing of the past.

Cassie Kozyrkov has served in various technical roles at Google over the past five years, but she now holds the somewhat curious position of "chief decision scientist." Decision science sits at the intersection of data and behavioral science and involves statistics, machine learning, psychology, economics, and more. In effect, this means Kozyrkov helps Google push a positive AI agenda -- or, at the very least, convince people that AI isn't as bad as the headlines claim. "Robots are stealing our jobs," "AI is humanity's greatest existential threat," and similar proclamations have abounded for a while, but over the past few years such fears have become more pronounced. Conversational AI assistants now live in our homes, cars and trucks are pretty much able to drive themselves, machines can beat humans at computer games, and even the creative arts are not immune to the AI onslaught. On the flip side, we're also told that boring and repetitive jobs could become a thing of the past.

While artificial intelligence was once heralded as the key to unlocking a new era of economic prosperity, policymakers today face a wave of calls to ensure AI is fair, ethical and safe. New York City Mayor de Blasio recently announced the formation of the nation's first task force to monitor and assess the use of algorithms. Days later, the European Union enacted sweeping new data protection rules that require companies be able to explain to consumers any automated decisions. And high-profile critics, like Elon Musk, have called on policymakers to do more to regulate AI. Unfortunately, the two most popular ideas -- requiring companies to disclose the source code to their algorithms and explain how they make decisions -- would cause more harm than good by regulating the business models and the inner workings of the algorithms of companies using AI, rather than holding these companies accountable for outcomes. The first idea -- "algorithmic transparency" -- would require companies to disclose the source code and data used in their AI systems.