Generative AI
What do people expect from Artificial Intelligence? Public opinion on alignment in AI moderation from Germany and the United States
Jungherr, Andreas, Rauchfleisch, Adrian
Recent advances in generative Artificial Intelligence have raised public awareness, shaping expectations and concerns about their societal implications. Central to these debates is the question of AI alignment -- how well AI systems meet public expectations regarding safety, fairness, and social values. However, little is known about what people expect from AI-enabled systems and how these expectations differ across national contexts. We present evidence from two surveys of public preferences for key functional features of AI-enabled systems in Germany (n = 1800) and the United States (n = 1756). We examine support for four types of alignment in AI moderation: accuracy and reliability, safety, bias mitigation, and the promotion of aspirational imaginaries. U.S. respondents report significantly higher AI use and consistently greater support for all alignment features, reflecting broader technological openness and higher societal involvement with AI. In both countries, accuracy and safety enjoy the strongest support, while more normatively charged goals -- like fairness and aspirational imaginaries -- receive more cautious backing, particularly in Germany. We also explore how individual experience with AI, attitudes toward free speech, political ideology, partisan affiliation, and gender shape these preferences. AI use and free speech support explain more variation in Germany. In contrast, U.S. responses show greater attitudinal uniformity, suggesting that higher exposure to AI may consolidate public expectations. These findings contribute to debates on AI governance and cross-national variation in public preferences. More broadly, our study demonstrates the value of empirically grounding AI alignment debates in public attitudes and of explicitly developing normatively grounded expectations into theoretical and policy discussions on the governance of AI-generated content.
"It Listens Better Than My Therapist": Exploring Social Media Discourse on LLMs as Mental Health Tool
Abstract: The emergence of generative AI chatbots such as ChatGPT has prompted growing public and academic interest in their role as informal mental health support tools. While early rule-based systems have been around since several years, large language models (LLMs) offer new capabilities in conversational fluency, empathy simulation, and availability. This study explores how users engage with LLMs as mental health tools by analyzing over 10,000 TikTok comments from videos referencing LLMs as mental health tools. Using a self-developed tiered coding schema and supervised classification models, we identify user experiences, attitudes, and recurring themes. Results show that nearly 20% of comments reflect personal use, with these users expressing overwhelmingly positive attitudes. Commonly cited benefits include accessibility, emotional support, and perceived therapeutic value. However, concerns around privacy, generic responses, and the lack of professional oversight remain prominent. It Is important to note that the user feedback does not indicate which therapeutic framework, if any, the LLM-generated output aligns with. While the findings underscore the growing relevance of AI in everyday practices, they also highlight the urgent need for clinical and ethical scrutiny in the use of AI for mental health support. This study does not endorse or encourage the use of AI tools as substitutes for professional mental health support.
Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability
Haase, Jennifer, Hanel, Paul H. P., Pokutta, Sebastian
Following the widespread adoption of ChatGPT in early 2023, numerous studies reported that large language models (LLMs) can match or even surpass human performance in creative tasks. However, it remains unclear whether LLMs have become more creative over time, and how consistent their creative output is. In this study, we evaluated 14 widely used LLMs -- including GPT-4, Claude, Llama, Grok, Mistral, and DeepSeek -- across two validated creativity assessments: the Divergent Association Task (DAT) and the Alternative Uses Task (AUT). Contrary to expectations, we found no evidence of increased creative performance over the past 18-24 months, with GPT-4 performing worse than in previous studies. For the more widely used AUT, all models performed on average better than the average human, with GPT-4o and o3-mini performing best. However, only 0.28% of LLM-generated responses reached the top 10% of human creativity benchmarks. Beyond inter-model differences, we document substantial intra-model variability: the same LLM, given the same prompt, can produce outputs ranging from below-average to original. This variability has important implications for both creativity research and practical applications. Ignoring such variability risks misjudging the creative potential of LLMs, either inflating or underestimating their capabilities. The choice of prompts affected LLMs differently. Our findings underscore the need for more nuanced evaluation frameworks and highlight the importance of model selection, prompt design, and repeated assessment when using Generative AI (GenAI) tools in creative contexts.
Multi-Stakeholder Disaster Insights from Social Media Using Large Language Models
Belcastro, Loris, Cosentino, Cristian, Marozzo, Fabrizio, Gรผndรผz-Cรผre, Merve, รztรผrk-Birim, Sule
In recent years, social media has emerged as a primary channel for users to promptly share feedback and issues during disasters and emergencies, playing a key role in crisis management. While significant progress has been made in collecting and analyzing social media content, there remains a pressing need to enhance the automation, aggregation, and customization of this data to deliver actionable insights tailored to diverse stakeholders, including the press, police, EMS, and firefighters. This effort is essential for improving the coordination of activities such as relief efforts, resource distribution, and media communication. This paper presents a methodology that leverages the capabilities of LLMs to enhance disaster response and management. Our approach combines classification techniques with generative AI to bridge the gap between raw user feedback and stakeholder-specific reports. Social media posts shared during catastrophic events are analyzed with a focus on user-reported issues, service interruptions, and encountered challenges. We employ full-spectrum LLMs, using analytical models like BERT for precise, multi-dimensional classification of content type, sentiment, emotion, geolocation, and topic. Generative models such as ChatGPT are then used to produce human-readable, informative reports tailored to distinct audiences, synthesizing insights derived from detailed classifications. We compare standard approaches, which analyze posts directly using prompts in ChatGPT, to our advanced method, which incorporates multi-dimensional classification, sub-event selection, and tailored report generation. Our methodology demonstrates superior performance in both quantitative metrics, such as text coherence scores and latent representations, and qualitative assessments by automated tools and field experts, delivering precise insights for diverse disaster response stakeholders.
If I Don't Use AI, Will My Grandkids Still Think I'm Cool?
As a retiree, I want to stay close to my grandkids. I worry that not learning how to use AI will leave me behind. What's the easiest tool for me to learn, and should I be worried? I promise that you do not need to learn how to use a generative AI tool like ChatGPT or Claude to ensure your grandkids see you as a relevant, informed person. If anything, I would say that our culture has tipped over the past year to generally oppose the use of generative AI tools due to their outsize environmental impact, ethical concerns over their data scraping, and general sludginess of the outputs.
Evaluating the Goal-Directedness of Large Language Models
Everitt, Tom, Garbacea, Cristina, Bellot, Alexis, Richens, Jonathan, Papadatos, Henry, Campos, Simรฉon, Shah, Rohin
To what extent do LLMs use their capabilities towards their given goal? We take this as a measure of their goal-directedness. We evaluate goal-directedness on tasks that require information gathering, cognitive effort, and plan execution, where we use subtasks to infer each model's relevant capabilities. Our evaluations of LLMs from Google DeepMind, OpenAI, and Anthropic show that goal-directedness is relatively consistent across tasks, differs from task performance, and is only moderately sensitive to motivational prompts. Notably, most models are not fully goal-directed. We hope our goal-directedness evaluations will enable better monitoring of LLM progress, and enable more deliberate design choices of agentic properties in LLMs.
Agile Retrospectives: What went well? What didn't go well? What should we do?
Spichkova, Maria, Lee, Hina, Iwan, Kevin, Zwart, Madeleine, Yoon, Yuwon, Qin, Xiaohan
In Agile/Scrum software development, the idea of retrospective meetings (retros) is one of the core elements of the project process. In this paper, we present our work in progress focusing on two aspects: analysis of potential usage of generative AI for information interaction within retrospective meetings, and visualisation of retros' information to software development teams. We also present our prototype tool RetroAI++, focusing on retros-related functionalities.
Can GPT tell us why these images are synthesized? Empowering Multimodal Large Language Models for Forensics
He, Yiran, Cao, Yun, Yang, Bowen, Zhang, Zeyu
The rapid development of generative AI facilitates content creation and makes image manipulation easier and more difficult to detect. While multimodal Large Language Models (LLMs) have encoded rich world knowledge, they are not inherently tailored for combating AI-generated Content (AIGC) and struggle to comprehend local forgery details. In this work, we investigate the application of multimodal LLMs in forgery detection. We propose a framework capable of evaluating image authenticity, localizing tampered regions, providing evidence, and tracing generation methods based on semantic tampering clues. Our method demonstrates that the potential of LLMs in forgery analysis can be effectively unlocked through meticulous prompt engineering and the application of few-shot learning techniques. We conduct qualitative and quantitative experiments and show that GPT4V can achieve an accuracy of 92.1% in Autosplice and 86.3% in LaMa, which is competitive with state-of-the-art AIGC detection methods. We further discuss the limitations of multimodal LLMs in such tasks and propose potential improvements.
OpenAI's new o3 and o4-mini models are all about 'thinking with images'
OpenAI is releasing not one but two new models. The company today announced the public availability of o3 and o4-mini. Of the former, OpenAI says o3 is its most advanced reasoning model yet, with it showing "strong performance" in coding, math and science tasks. As for o4-mini, OpenAI is billing it as a lower cost alternative that still delivers "impressive results" across those same fields. More notably, both models offer novel capabilities not found in OpenAI's past systems.
Early Impacts of M365 Copilot
Dillon, Eleanor Wiske, Jaffe, Sonia, Peng, Sida, Cambon, Alexia
New generative AI tools have been developing rapidly and are now widely used, including by workers in doing their jobs. Microsoft worked with firms across industries to run a large field experiment to measure how access to one of these tools changes work patterns. The experiment ran during the early rollout of Microsoft's M365 Copilot (Copilot), a multi-part generative AI tool that integrates generative AI into components of Microsoft's M365 suite (including Word, PowerPoint, Outlook, and Teams). M365 Copilot is designed as a general purpose tool to help workers digest information by summarizing emails, meetings, or documents, create new content by drafting emails, documents, or slide decks, and retrieve information either from the web or across any company email, chat, or document to which the worker has access. We worked with firms to randomize access to Copilot and got permission to use several months of anonymized metadata on workers' behaviors in Outlook, Teams, and Office, both before and after access to Copilot.