Requirements Engineering Framework for Human-centered Artificial Intelligence Software Systems

AI-based software systems are rapidly becoming essential in many organizations [1]. However, the focus on the technical side of building artificial intelligence (AI)-based systems are most common, and many projects, more often than not, fail to address critical human aspects during the development phases [2, 3]. These include but are not limited to age, gender, ethnicity, socio-economic status, education, language, culture, emotions, personality, and many others [4]. Ignoring human-centered aspects in AI-based software tends to produce biased and non-inclusive outcomes [5]. Shneiderman [6] emphasizes the dangers of autonomy-first design in AI and the hidden biases that follow. Misrepresenting human aspects in requirements for model selection and data used in training AI algorithms can lead to discriminatory decision procedures even if the underlying computational processes were unbiased [7]. For example, a study by Carnegie Mellon revealed that women were far less likely to receive high-paying job ads from Google than men [8] due to the under-representation of people of color and women in high paying IT jobs. Studies on human-centered design aim to develop systems that put human needs and values at the center of software development and clearly understand the context of the software system's usage [2, 9].