Unveiling Biases while Embracing Sustainability: Assessing the Dual Challenges of Automatic Speech Recognition Systems

Unveiling Biases while Embracing Sustainability: Assessing the Dual Challenges of Automatic Speech Recognition Systems Ajinkya Kulkarni 1, 2, Atharva Kulkarni 3, Miguel Couceiro 4, 5, Isabel Trancoso 5 1 IDIAP, Switzerland, 2 MBZUAI, UAE, 3 Erisha Labs, India 4 Universit e de Lorraine, CNRS, LORIA, Nancy, France 5 INESC-ID, IST, Universidade de Lisboa, Portugal ajinkya.kulkarni@idiap.ch Abstract In this paper, we present a bias and sustainability focused investigation of Automatic Speech Recognition (ASR) systems, namely Whisper and Massively Multilingual Speech (MMS), which have achieved state-of-the-art (SOT A) performances. Despite their improved performance in controlled settings, there remains a critical gap in understanding their efficacy and equity in real-world scenarios. In addition, we examine the environmental impact of ASR systems, scrutinizing the use of large acoustic models on carbon emission and energy consumption. We also provide insights into our empirical analyses, offering a valuable contribution to the claims surrounding bias and sustainability in ASR systems. Index T erms: ASR, Bias, carbon footprint, sustainability 1. Introduction The advent of large deep neural networks (DNNs) has brought about substantial advancements in various speech-processing applications, notably in speech recognition.