Fairlearn is an open source project to help practitioners assess and improve fairness of artificial intelligence (AI) systems. The associated Python library, also named fairlearn, supports evaluation of a model's output across affected populations and includes several algorithms for mitigating fairness issues. Grounded in the understanding that fairness is a sociotechnical challenge, the project integrates learning resources that aid practitioners in considering a system's broader societal context.

In the field of computer science, large-scale experimentation on users is not new. However, driven by advances in artificial intelligence, novel autonomous systems for experimentation are emerging that raise complex, unanswered questions for the field. Some of these questions are computational, while others relate to the social and ethical implications of these systems. We see these normative questions as urgent because they pertain to critical infrastructure upon which large populations depend, such as transportation and healthcare. Although experimentation on widely used online platforms like Facebook has stoked controversy in recent years, the unique risks posed by autonomous experimentation have not received sufficient attention, even though such techniques are being trialled on a massive scale. In this paper, we identify several questions about the social and ethical implications of autonomous experimentation systems.

Today we're joined by Hanna Wallach, a Principal Researcher at Microsoft Research. We discuss the role that human biases, even those that are inadvertent, play in tainting data, and whether deployment of "fair" ML models can actually be achieved in practice, and much more. Along the way, Hanna points us to a TON of papers and resources to further explore the topic of fairness in ML. You'll definitely want to check out the notes page for this episode, which you'll find at twimlai.com/talk/232. We'd like to thank Microsoft for their support and their sponsorship of this series.

Since the 1980s the number of women completing computer science degrees has plummeted, and in most large tech companies the representation of women in technical roles is below 30%. This lack of diversity prevents us from building products that work for everybody. It can foster toxic "brogrammer" cultures which harm everybody who works within them, and it deprives teams of the well-documented performance boost that women bring. Many of the early superstars in computer science were women -- from Lord Byron's polymath daughter Ada Lovelace, the first person to envisage a general purpose computer, to Rear Admiral Grace Hooper, who pioneered the use of natural language in writing computer programs. Similarly, the post-war computing scene was dominated by women.

In this talk, I will introduce the audience to the emerging area of computational social science, focusing on how machine learning for social science differs from machine learning in other contexts. I will present two related models -- both based on Bayesian Poisson tensor decomposition -- for uncovering latent structure from count data. The first is for uncovering topics in previously classified government documents, while the second is for uncovering multilateral relations from country-to-country interaction data. Finally, I will talk briefly about the broader ethical implications of analyzing social data. Hanna Wallach is a Senior Researcher at Microsoft Research New York City and an Adjunct Associate Professor in the College of Information and Computer Sciences at the University of Massachusetts Amherst.

In episode four we talk with Hanna Wallach, of Microsoft Research. We take a listener question about scalability and the size of data sets. And Ryan takes us through topic modeling using Latent Dirichlet allocation (say that five times fast). See all the latest robotics news on Robohub, or sign up for our weekly newsletter.