How Large Language Models play humans in online conversations: a simulated study of the 2016 US politics on Reddit

--Large Language Models (LLMs) have recently emerged as powerful tools for natural language generation, with applications spanning from content creation to social simulations. Their ability to mimic human interactions raises both opportunities and concerns, particularly in the context of politically relevant online discussions. In this study, we evaluate the performance of LLMs in replicating user-generated content within a real-world, divisive scenario: Reddit conversations during the 2016 US Presidential election. In particular, we conduct three different experiments, asking GPT -4 to generate comments by impersonating either real or artificial partisan users. We analyze the generated comments in terms of political alignment, sentiment, and linguistic features, comparing them against real user contributions and benchmarking against a null model. We find that GPT -4 is able to produce realistic comments, both in favor of or against the candidate supported by the community, yet tending to create consensus more easily than dissent. In addition we show that real and artificial comments are well separated in a semantically embedded space, although they are indistinguishable by manual inspection. Our findings provide insights on the potential use of LLMs to sneak into online discussions, influence political debate and shape political narratives, bearing broader implications of AI-driven discourse manipulation. Artificial intelligence (AI) has been the cornerstone of scientific inquiry and technological advancement for several decades, driving innovation in multiple scientific fields [1]. Despite its long-standing presence, AI has captured unprecedented public and academic attention in recent years, largely due to breakthroughs in generative models [2]. Among these, Large Language Models (LLMs) [3] stand out as a transforma-tive innovation, redefining how we approach problems in natural language processing, decision-making, and simulations. In the past two years, the release of powerful models capable of generating coherent and contextually relevant responses (such as GPT -3.5 and GPT -4 [4], Llama [5], Mistral [6] and Gemini [7]) not only captivated the public imagination, but also opened new avenues for research in complex systems [8]-[10]. In particular, LLMs have sparked a lot of interest in complex networks studies and Agent-Based models (ABM). For example, a population of interacting LLMs agents was shown to exhibit preferential attachment [11] and thus creating scale-free networks [12], a characteristic found in many real-world systems [13].