Machine learning (ML) models are increasingly used for personnel assessment and selection (e.g., resume screeners, automatically scored interviews). However, concerns have been raised throughout society that ML assessments may be biased and perpetuate or exacerbate inequality. Although organizational researchers have begun investigating ML assessments from traditional psychometric and legal perspectives, there is a need to understand, clarify, and integrate fairness operationalizations and algorithmic bias mitigation methods from the computer science, data science, and organizational research literatures. We present a four-stage model of developing ML assessments and applying bias mitigation methods, including 1) generating the training data, 2) training the model, 3) testing the model, and 4) deploying the model. When introducing the four-stage model, we describe potential sources of bias and unfairness at each stage. Then, we systematically review definitions and operationalizations of algorithmic bias, legal requirements governing personnel selection from the United States and Europe, and research on algorithmic bias mitigation across multiple domains and integrate these findings into our framework. Our review provides insights for both research and practice by elucidating possible mechanisms of algorithmic bias while identifying which bias mitigation methods are legal and effective. This integrative framework also reveals gaps in the knowledge of algorithmic bias mitigation that should be addressed by future collaborative research between organizational researchers, computer scientists, and data scientists. We provide recommendations for developing and deploying ML assessments, as well as recommendations for future research into algorithmic bias and fairness.

Model multiplicity, the phenomenon where multiple models achieve similar performance despite different underlying learned functions, introduces arbitrariness in model selection. While this arbitrariness may seem inconsequential in expectation, its impact on individuals can be severe. This paper explores various individual concerns stemming from multiplicity, including the effects of arbitrariness beyond final predictions, disparate arbitrariness for individuals belonging to protected groups, and the challenges associated with the arbitrariness of a single algorithmic system creating a monopoly across various contexts. It provides both an empirical examination of these concerns and a comprehensive analysis from the legal standpoint, addressing how these issues are perceived in the anti-discrimination law in Canada. We conclude the discussion with technical challenges in the current landscape of model multiplicity to meet legal requirements and the legal gap between current law and the implications of arbitrariness in model selection, highlighting relevant future research directions for both disciplines.

This paper describes a generalizable model evaluation method that can be adapted to evaluate AI/ML models across multiple criteria including core scientific principles and more practical outcomes. Emerging from prediction competitions in Psychology and Decision Science, the method evaluates a group of candidate models of varying type and structure across multiple scientific, theoretic, and practical criteria. Ordinal ranking of criteria scores are evaluated using voting rules from the field of computational social choice and allow the comparison of divergent measures and types of models in a holistic evaluation. Additional advantages and applications are discussed.

All types of research, development, and policy work can have unintended, adverse consequences - work in responsible artificial intelligence (RAI), ethical AI, or ethics in AI is no exception.

Organizations are increasingly adopting machine learning (ML) for personnel assessment. However, concerns exist about fairness in designing and implementing ML assessments. Supervised ML models are trained to model patterns in data, meaning ML models tend to yield predictions that reflect subgroup differences in applicant attributes in the training data, regardless of the underlying cause of subgroup differences. In this study, we systematically under- and oversampled minority (Black and Hispanic) applicants to manipulate adverse impact ratios in training data and investigated how training data adverse impact ratios affect ML model adverse impact and accuracy. We used self-reports and interview transcripts from job applicants (N = 2,501) to train 9,702 ML models to predict screening decisions. We found that training data adverse impact related linearly to ML model adverse impact. However, removing adverse impact from training data only slightly reduced ML model adverse impact and tended to negatively affect ML model accuracy. We observed consistent effects across self-reports and interview transcripts, whether oversampling real (i.e., bootstrapping) or synthetic observations. As our study relied on limited predictor sets from one organization, the observed effects on adverse impact may be attenuated among more accurate ML models.

In this #4 post of the Symposium "Hitchhikers Guide to Law & Tech", Nathalie Smuha and Anna Morandini continue asking fundamental questions on the interaction between regulation and technology. Can the European AI Act mitigate the ethical and legal concerns raised by this hyped technology? Which trail is the EU blazing to secure "Trustworthy Artificial Intelligence" in Europe, as distinct from the laissez-faire approach in the US and the state-centric approach in China? In this post, both authors unpack the proposed AI regulation and evaluate its merits and pitfalls. After explaining the build-up towards the proposal, they set out the scope of the Act and its four categories of risks as part of a "risk-based approach" to regulate AI.

In 2023, a new law regulating AI-enabled recruiting will go live in New York City, with more legislatures to inevitably follow. This is nearly a decade after Amazon deployed its infamous AI-recruiting tool that caused harmful bias against female candidates. Emerging technologies are often left unchecked as industries take shape around them. Due to rapid innovation and sluggish regulation, first-to-market companies tend to ask for the public's forgiveness versus seeking institutional permission. Nearly 20 years after its founding, Facebook (now Meta) is still largely self-regulated.

In November 2021 the All-Party Parliamentary Group ("APPG") on the Future of Work ("Future of Work") published its report titled "The New Frontier: Artificial Intelligence at Work" (the "Report"). The Report follows the National AI Strategy (the "Strategy") released by the government in September and sets out to identify and resolve challenges posed by artificial intelligence ("AI") in the workplace through the development of a new regulatory framework. Whilst the proposed framework addresses AI in the workforce, we consider some of the principles could be applied across all sectors. The recommendations made by the Future of Work inform the wider debate about AI governance and regulation as part of the Strategy. APPGs are informal cross-party groups that have no official status in Parliament but are run by and for Members of the Commons and Lords, bringing together parliamentarians, industry and civil society. There is an Artificial Intelligence APPG, but the author of the Report is the Future of Work, an APPG which aims to "foster understanding of the challenges and opportunities of technology and the future of work".