Beyond Recommender: An Exploratory Study of the Effects of Different AI Roles in AI-Assisted Decision Making

Artificial Intelligence (AI) is increasingly employed in various decisionmaking However, empirical research reveals several limitations within tasks, typically as a Recommender, providing recommendations the existing AI-assisted decision-making framework, wherein AI that the AI deems correct. However, recent studies suggest this acts primarily as a recommender. One notable issue is that individuals, may diminish human analytical thinking and lead to humans' inappropriate when passively receiving AI suggestions, seldom engage reliance on AI, impairing the synergy in human-AI teams. in analytical thinking [3, 7, 38]. Furthermore, people frequently In contrast, human advisors in group decision-making perform inappropriately rely on the AI's recommendations (such as overreliance various roles, such as analyzing alternative options or criticizing and under-reliance) [8, 30, 33, 46] and the mere provision of decision-makers to encourage their critical thinking. This diversity AI explanations can, paradoxically, exacerbate overreliance [2, 37]. of roles has not yet been empirically explored in AI assistance. In In comparison, in human-human decision-making, beyond recommenders, this paper, we examine three AI roles: Recommender, Analyzer, and human advisors sometimes play other types of roles, Devil's Advocate, and evaluate their effects across two AI performance such as helping the decision-makers analyze the pros and cons of levels. Our results show each role's distinct strengths and different alternatives instead of directly giving recommendations, or limitations in task performance, reliance appropriateness, and user critically challenging decision-makers' initial views [40, 42].