AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context
Rizvi, Naba, Strickland, Harper, Gitelman, Daniel, Cooper, Tristan, Morales-Flores, Alexis, Golden, Michael, Kallepalli, Aekta, Alurkar, Akshat, Owens, Haaset, Ahmedi, Saleha, Khirwadkar, Isha, Munyaka, Imani, Ousidhoum, Nedjma
–arXiv.org Artificial Intelligence
As our understanding of autism and ableism continues to increase, so does our understanding of ableist language towards autistic people. Such language poses a significant challenge in NLP research due to its subtle and context-dependent nature. Yet, detecting anti-autistic ableist language remains underexplored, with existing NLP tools often failing to capture its nuanced expressions. We present AUTALIC, the first benchmark dataset dedicated to the detection of anti-autistic ableist language in context, addressing a significant gap in the field. The dataset comprises 2,400 autism-related sentences collected from Reddit, accompanied by surrounding context, and is annotated by trained experts with backgrounds in neurodiversity. Our comprehensive evaluation reveals that current language models, including state-of-the-art LLMs, struggle to reliably identify anti-autistic ableism and align with human judgments, underscoring their limitations in this domain. We publicly release AUTALIC along with the individual annotations which serve as a valuable resource to researchers working on ableism, neurodiversity, and also studying disagreements in annotation tasks. This dataset serves as a crucial step towards developing more inclusive and context-aware NLP systems that better reflect diverse perspectives.
arXiv.org Artificial Intelligence
Nov-21-2024
- Country:
- Asia
- Indonesia > Bali (0.04)
- Middle East
- Israel (0.04)
- Jordan (0.04)
- Saudi Arabia > Asir Province
- Abha (0.04)
- UAE > Abu Dhabi Emirate
- Abu Dhabi (0.04)
- Europe
- North America
- Canada > Ontario
- Toronto (0.04)
- United States
- California > San Diego County
- San Diego (0.04)
- Louisiana > Orleans Parish
- New Orleans (0.04)
- New York > New York County
- New York City (0.04)
- California > San Diego County
- Canada > Ontario
- Asia
- Genre:
- Research Report
- Experimental Study (0.68)
- New Finding (0.46)
- Research Report
- Industry:
- Health & Medicine > Therapeutic Area > Neurology > Autism (1.00)
- Technology: