Embracing Dialectic Intersubjectivity: Coordination of Different Perspectives in Content Analysis with LLM Persona Simulation

Kang, Taewoo, Thorson, Kjerstin, Peng, Tai-Quan, Hiaeshutter-Rice, Dan, Lee, Sanguk, Soroka, Stuart

arXiv.org Artificial Intelligence 

Large Language Models (LLMs), such as ChatGPT, allow researchers to analyze extensive text corpora with efficiency (Bail, 2024). However, human and algorithmic biases can influence their outputs, reflecting ideological or cultural skewness in the training data (Bender et al., 2021; Kroon et al., 2023). With the growing adoption of LLM-Assisted Content Analysis (LACA)--a method replacing manual coding with LLM-generated datasets (Chew et al., 2023)--the scientific community must ensure these tools enhance understanding rather than reinforce existing biases (Messeri and Crockett, 2024). Human bias is a long-standing issue in content analysis, which traditionally depends on a consensus-oriented research practice. Intercoder agreement is used to ensure reliability, aiming for shared interpretations of text that can be statistically validated (Krippendorff, 1999). However, this emphasis on consensus may inadvertently neglect diverse perspectives, favoring uniform interpretations that risk simplifying the sociocultural complexities embedded in textual connotations.

Duplicate Docs Excel Report

Title
None found

Similar Docs  Excel Report  more

TitleSimilaritySource
None found