We know that language models (LMs) form biases and stereotypes of minorities, leading to unfair treatments of members of these groups, thanks to research mainly in the US and the broader English-speaking world. As the negative behavior of these models has severe consequences for society and individuals, industry and academia are actively developing methods to reduce the bias in LMs. However, there are many under-represented groups and languages that have been overlooked so far. This includes marginalized groups that are specific to individual countries and regions in the English speaking and Western world, but crucially also almost all marginalized groups in the rest of the world. The UN estimates, that between 600 million to 1.2 billion people worldwide are members of marginalized groups and in need for special protection. If we want to develop inclusive LMs that work for everyone, we have to broaden our understanding to include overlooked marginalized groups and low-resource languages and dialects. In this work, we contribute to this effort with the first study investigating offensive stereotyping bias in 23 LMs for 270 marginalized groups from Egypt, the remaining 21 Arab countries, Germany, the UK, and the US. Additionally, we investigate the impact of low-resource languages and dialects on the study of bias in LMs, demonstrating the limitations of current bias metrics, as we measure significantly higher bias when using the Egyptian Arabic dialect versus Modern Standard Arabic. Our results show, LMs indeed show higher bias against many marginalized groups in comparison to dominant groups. However, this is not the case for Arabic LMs, where the bias is high against both marginalized and dominant groups in relation to religion and ethnicity. Our results also show higher intersectional bias against Non-binary, LGBTQIA+ and Black women.

Markowicz previously explained why they wrote the book in a Fox News Digital opinion piece, noting that in 2021, then-Democratic Virginia gubernatorial candidate Terry McAuliffe said, "I don't think parents should be telling schools what they should teach." "Taken on its own, the comment might even be benign. Sure, parental involvement in education had always been a prediction of student success. A 2010 study called'Parent Involvement and Student Academic Performance: A Multiple Mediational Analysis' by researchers at the Warren Alpert Medical School of Brown University and the University of North Carolina at Greensboro found'children whose parents are more involved in their education have higher levels of academic performance than children whose parents are involved to a lesser degree." But should parents be designing a curriculum?

Commercial content moderation APIs are marketed as scalable solutions to combat online hate speech. However, the reliance on these APIs risks both silencing legitimate speech, called over-moderation, and failing to protect online platforms from harmful speech, known as under-moderation. To assess such risks, this paper introduces a framework for auditing black-box NLP systems. Using the framework, we systematically evaluate five widely used commercial content moderation APIs. Analyzing five million queries based on four datasets, we find that APIs frequently rely on group identity terms, such as ``black'', to predict hate speech. While OpenAI's and Amazon's services perform slightly better, all providers under-moderate implicit hate speech, which uses codified messages, especially against LGBTQIA+ individuals. Simultaneously, they over-moderate counter-speech, reclaimed slurs and content related to Black, LGBTQIA+, Jewish, and Muslim people. We recommend that API providers offer better guidance on API implementation and threshold setting and more transparency on their APIs' limitations. Warning: This paper contains offensive and hateful terms and concepts. We have chosen to reproduce these terms for reasons of transparency.

Although large language models (LLMs) demonstrate impressive proficiency in various tasks, they present potential safety risks, such as `jailbreaks', where malicious inputs can coerce LLMs into generating harmful content bypassing safety alignments. In this paper, we delve into the ethical biases in LLMs and examine how those biases could be exploited for jailbreaks. Notably, these biases result in a jailbreaking success rate in GPT-4o models that differs by 20\% between non-binary and cisgender keywords and by 16\% between white and black keywords, even when the other parts of the prompts are identical. We introduce the concept of BiasJailbreak, highlighting the inherent risks posed by these safety-induced biases. BiasJailbreak generates biased keywords automatically by asking the target LLM itself, and utilizes the keywords to generate harmful output. Additionally, we propose an efficient defense method BiasDefense, which prevents jailbreak attempts by injecting defense prompts prior to generation. BiasDefense stands as an appealing alternative to Guard Models, such as Llama-Guard, that require additional inference cost after text generation. Our findings emphasize that ethical biases in LLMs can actually lead to generating unsafe output, and suggest a method to make the LLMs more secure and unbiased. To enable further research and improvements, we open-source our code and artifacts of BiasJailbreak, providing the community with tools to better understand and mitigate safety-induced biases in LLMs.

While traditional discussions of epistemic injustice have While algorithms have traditionally been leveraged to primarily centered on interpersonal human interactions present and organize human-generated content, the advent (McKinnon 2017; Tsosie 2012), existing research on algorithmic of generative AI has started to fundamentally shift this epistemic injustice has largely been limited to epistemic paradigm. Generative AI models can now create content - injustices produced by decision-making and classification spanning text, imagery, and beyond - that resembles that of algorithms. However, we argue that the distinctive authors, journalists, painters, or photographers. In this paper, characteristics of generative AI give rise to novel forms of we take generative AI to be the class of machine learning epistemic injustice that necessitate a dedicated analytical models trained on massive amounts of data, typically media framework. To address this, we expand upon the established such as text, images, audio or video, in order to produce philosophical discourse on epistemic injustice and introduce representative instances of such media (García-Peñalvo and an account of "generative algorithmic epistemic injustice," Vázquez-Ingelmo 2023).

The rise of foundation models holds immense promise for advancing AI, but this progress may amplify existing risks and inequalities, leaving marginalized communities behind. In this position paper, we discuss that disparities towards marginalized communities - performance, representation, privacy, robustness, interpretability and safety - are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We contrast foundation models with traditional models and highlight the potential for exacerbated disparity against marginalized communities. Moreover, we emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences, specifically to the people on the margin. We define marginalized communities within the machine learning context and explore the multifaceted nature of disparities. We analyze the sources of these disparities, tracing them from data creation, training and deployment procedures to highlight the complex technical and socio-technical landscape. To mitigate the pressing crisis, we conclude with a set of calls to action to mitigate disparity at its source.

Then, I address the identified research problems Hate speech on social media has severe negative in hate speech detection, by investigating the impacts, not only on its victims (Sticca et al., impact of bias in NLP models on hate speech 2013) but also on the moderators of social detection models from three perspectives: 1) the media platforms (Roberts, 2019). This is why explainability perspective ( 4), where I address the it is crucial to develop tools for automated hate first research problem and investigate the impact speech detection. These tools should provide of bias in NLP models on their performance of a safer environment for individuals, especially hate speech detection and whether the bias in for members of marginalized groups, to express NLP models explains their performance on hate themselves online. However, recent research shows speech detection; 2) the offensive stereotyping that current hate speech detection models falsely bias perspective ( 5), where I address the second flag content written by members of marginalized research problem and investigate the impact of communities, as hateful (Sap et al., 2019; Dixon imbalanced representations and co-occurrences of et al., 2018; Mchangama et al., 2021). Similarly, hateful content with marginalized identity groups recent research indicates that there are social biases on the bias of NLP models; and 3) the fairness in natural language processing (NLP) models (Garg perspective ( 6), where I address the third research et al., 2018; Nangia et al., 2020; Kurita et al., 2019; problem and investigate the impact of bias in Ousidhoum et al., 2021; Nozza et al., 2021, 2022). NLP models on the fairness of the task of hate Yet, the impact of these biases on the task of speech detection. For each research problem, I hate speech detection has been understudied. In summarize the work done to highlight its main my thesis, I identify and study three research findings, contributions, and limitations. Thereafter, problems: 1) the impact of bias in NLP models on I discuss the general takeaways from my thesis and the performance and explainability of hate speech how it can benefit the NLP community at large ( 7).

Research has shown that language models (LMs) are socially biased. However, toxicity and offensive stereotyping bias in LMs are understudied. In this paper, we investigate the systematic offensive stereotype (SOS) bias in LMs. We propose a method to measure it. Then, we validate the SOS bias and investigate the effectiveness of debias methods from the literature on removing it. Finally, we investigate the impact of the SOS bias in LMs on their performance and their fairness on the task of hate speech detection. Our results suggest that all the inspected LMs are SOS biased. The results suggest that the SOS bias in LMs is reflective of the hate experienced online by the inspected marginalized groups. The results indicate that removing the SOS bias in LMs, using a popular debias method from the literature, leads to worse SOS bias scores. Finally, Our results show no strong evidence that the SOS bias in LMs is impactful on their performance on hate speech detection. On the other hand, there is evidence that the SOS bias in LMs is impactful on their fairness.

Every time a new application of AI is announced, I feel a short-lived rush of excitement -- followed soon after by a knot in my stomach. This is because I know the technology, more often than not, hasn't been designed with equity in mind. One system, ChatGPT, has reached 100 million unique users just two months after its launch. The text-based tool engages users in interactive, friendly, AI-generated exchanges with a chatbot that has been developed to speak authoritatively on any subject it's prompted to address. In an interview with Michael Barbaro on the The Daily podcast from the New York Times, tech reporter Kevin Roose described how an app similar to ChatGPT, Bing's AI chatbot, which also is built on OpenAI's GPT-3 language model, responded to his request for a suggestion on a side dish to accompany French onion soup for Valentine's Day dinner with his wife.

Jazmia Henry July 18, 2022 Summary Discussions about reducing the bias present in algorithms have been on the rise since the mid 2010s. AI ethicists, DEI practitioners, Sociologists, Data Scientists and Social Justice Advocates have decried the lack of understanding of the harms that algorithms pose to people who belong to historically marginalized groups. These cries have become increasingly accepted in industry since 2020, but little is understood of how algorithm and Machine Learning (ML) model builders should go about mitigating bias in models that are intended for deployment. This chapter is written with the Data Scientist or MLOps professional in mind but can be used as a resource for policy makers, reformists, AI Ethicists, sociologists, and others interested in finding methods that help reduce bias in algorithms. I will take a deployment centered approach with the assumption that the professionals reading this work have already read the amazing work on the implications of algorithms on historically marginalized groups by Gebru, Buolamwini, Benjamin and Shane to name a few. If you have not read those works, I refer you to the "Important Reading for Ethical Model Building " list at the end of this paper as it will help give you a framework on how to think about Machine Learning models more holistically taking into account their effect on marginalized people. In the Introduction to this chapter, I root the significance of their work in real world examples of what happens when models are deployed without transparent data collected for the training process and are deployed without the practitioners paying special attention to what happens to models that adapt to exploit gaps between their training environment and the real world. The rest of this chapter builds on the work of the aforementioned researchers and discusses the reality of models performing post production and details ways ML practitioners can identify bias using tools during the MLOps lifecycle to mitigate bias that may be introduced to models in the real world. Introduction "Whether AI will help us reach our aspirations or reinforce the unjust inequalities is ultimately up to us." - Joy Buolowini, 'Facing the Coded Gaze' AI: More than Human Whether you're driving your car using a GPS system, call on Alexa or Siri to turn on your favorite tune, go on social media to perform a well-earned scroll down memory lane, or go to Google search to find a gift to buy for a friend, you have encountered a Machine Learning model.