Embracing Dialectic Intersubjectivity: Coordination of Different Perspectives in Content Analysis with LLM Persona Simulation
Kang, Taewoo, Thorson, Kjerstin, Peng, Tai-Quan, Hiaeshutter-Rice, Dan, Lee, Sanguk, Soroka, Stuart
–arXiv.org Artificial Intelligence
Large Language Models (LLMs), such as ChatGPT, allow researchers to analyze extensive text corpora with efficiency (Bail, 2024). However, human and algorithmic biases can influence their outputs, reflecting ideological or cultural skewness in the training data (Bender et al., 2021; Kroon et al., 2023). With the growing adoption of LLM-Assisted Content Analysis (LACA)--a method replacing manual coding with LLM-generated datasets (Chew et al., 2023)--the scientific community must ensure these tools enhance understanding rather than reinforce existing biases (Messeri and Crockett, 2024). Human bias is a long-standing issue in content analysis, which traditionally depends on a consensus-oriented research practice. Intercoder agreement is used to ensure reliability, aiming for shared interpretations of text that can be statistically validated (Krippendorff, 1999). However, this emphasis on consensus may inadvertently neglect diverse perspectives, favoring uniform interpretations that risk simplifying the sociocultural complexities embedded in textual connotations.
arXiv.org Artificial Intelligence
Feb-4-2025
- Country:
- North America > United States
- Michigan (0.04)
- Texas (0.04)
- Illinois (0.04)
- Colorado (0.04)
- California (0.04)
- North America > United States
- Genre:
- Research Report > New Finding (0.67)
- Industry:
- Media > News (0.70)
- Government
- Technology: