Goto

Collaborating Authors

 ethnicity and gender


Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations

arXiv.org Artificial Intelligence

Language models have been shown to propagate social bias through their output, particularly in the representation of gender and ethnicity. This paper investigates gender and ethnicity biases in AI-generated occupational stories. Representation biases are measured before and after applying our proposed mitigation strategy, Bias Analysis and Mitigation through Explanation (BAME), revealing improvements in demographic representation ranging from 2% to 20%. BAME leverages model-generated explanations to inform targeted prompt engineering, effectively reducing biases without modifying model parameters. By analyzing stories generated across 25 occupational groups, three large language models (Claude 3.5 Sonnet, Llama 3.1 70B Instruct, and GPT-4 Turbo), and multiple demographic dimensions, we identify persistent patterns of overrepresentation and underrepresentation linked to training data stereotypes. Our findings demonstrate that guiding models with their own internal reasoning mechanisms can significantly enhance demographic parity, thereby contributing to the development of more transparent generative AI systems.


Met urged to scrap Carnival facial recognition plan

BBC News

The letter also raised concerns over a 2023 National Physical Laboratory study, which found the NeoFace system used by the Met was less accurate for women and people of colour depending on the algorithm that has been set. The study's authors found the system could show bias at lower thresholds, though at the higher settings the Met says it uses, performance was found to be equitable across ethnicity and gender. These thresholds are confidence levels the system uses to decide a match - lower ones flag more people but risk more mistakes and bias, while higher ones are stricter and more balanced. Campaigners said there was no legal obligation for the force to avoid the lower thresholds, and argued policing resources would be better spent on safety measures at the carnival. Deputy Assistant Commissioner Matt Ward, who is leading this year's policing operation at the carnival, said LFR had led to more than 1,000 arrests since the start of 2024 and that independent testing showed the system was "accurate and balanced with regard to ethnicity and gender" at the thresholds used by the Met.


When bias in product design means life or death

#artificialintelligence

Carol E. Reiley is the co-founder and president of Drive.ai. She previously founded Tinkerbelle Laboratories. During my Ph.D. studies, I developed a voice-activated human-robot interface for a surgical robotic system using Microsoft's speech recognition API. But, because the API had been built mainly by 20-30-year-old men, it did not recognize my voice. I had to lower my pitch in order for it to work.


When bias in product design means life or death

#artificialintelligence

Carol E. Reiley is the co-founder and president of Drive.ai. She previously founded Tinkerbelle Laboratories. During my Ph.D. studies, I developed a voice-activated human-robot interface for a surgical robotic system using Microsoft's speech recognition API. But, because the API had been built mainly by 20-30-year-old men, it did not recognize my voice. I had to lower my pitch in order for it to work.