Personal
Dalí, Basquiat, Haring, and Hockney at Luna Luna
I don't know what Werner Herzog is up to these days, but if he's between projects, I humbly suggest that he make a documentary about Luna Luna, the Hamburg amusement park that took more than ten years to put together, included attractions designed by Dalí and Basquiat and Haring and Hockney, and spent thirty-five years in shipping containers. It's now been partly reassembled at the Shed, for the exhibition "Luna Luna, Forgotten Fantasy," through Jan. 5. The park's Fitzcarraldo, a poet-songwriter-pop star named André Heller, was born in Vienna in 1947 and spent much of his thirties persuading artists to decorate rides. Haring slathered a merry-go-round in melty cartoons; Basquiat dressed a Ferris wheel in his customary graffiti. The park opened to the public in 1987, largely funded by a gossip rag, and stayed that way for a summer.
Scaling up the Evaluation of Collaborative Problem Solving: Promises and Challenges of Coding Chat Data with ChatGPT
Hao, Jiangang, Cui, Wenju, Kyllonen, Patrick, Kerzabi, Emily, Liu, Lei, Flor, Michael
Collaborative problem solving (CPS) is widely recognized as a critical 21st century skill. Efficiently coding communication data is a big challenge in scaling up research on assessing CPS. This paper reports the findings on using ChatGPT to directly code CPS chat data by benchmarking performance across multiple datasets and coding frameworks. We found that ChatGPT-based coding outperformed human coding in tasks where the discussions were characterized by colloquial languages but fell short in tasks where the discussions dealt with specialized scientific terminology and contexts. The findings offer practical guidelines for researchers to develop strategies for efficient and scalable analysis of communication data from CPS tasks.
Exploratory Study Of Human-AI Interaction For Hindustani Music
Shikarpur, Nithya, Huang, Cheng-Zhi Anna
This paper presents a study of participants interacting with and using GaMaDHaNi, a novel hierarchical generative model for Hindustani vocal contours. To explore possible use cases in human-AI interaction, we conducted a user study with three participants, each engaging with the model through three predefined interaction modes. Although this study was conducted "in the wild"-- with the model unadapted for the shift from the training data to real-world interaction -- we use it as a pilot to better understand the expectations, reactions, and preferences of practicing musicians when engaging with such a model. We note their challenges as (1) the lack of restrictions in model output, and (2) the incoherence of model output. We situate these challenges in the context of Hindustani music and aim to suggest future directions for the model design to address these gaps.
NewsInterview: a Dataset and a Playground to Evaluate LLMs' Ground Gap via Informational Interviews
Lu, Michael, Cho, Hyundong Justin, Shi, Weiyan, May, Jonathan, Spangher, Alexander
Large Language Models (LLMs) have demonstrated impressive capabilities in generating coherent text but often struggle with grounding language and strategic dialogue. To address this gap, we focus on journalistic interviews, a domain rich in grounding communication and abundant in data. We curate a dataset of 40,000 two-person informational interviews from NPR and CNN, and reveal that LLMs are significantly less likely than human interviewers to use acknowledgements and to pivot to higher-level questions. Realizing that a fundamental deficit exists in multi-turn planning and strategic thinking, we develop a realistic simulated environment, incorporating source personas and persuasive elements, in order to facilitate the development of agents with longer-horizon rewards. Our experiments show that while source LLMs mimic human behavior in information sharing, interviewer LLMs struggle with recognizing when questions are answered and engaging persuasively, leading to suboptimal information extraction across model size and capability. These findings underscore the need for enhancing LLMs' strategic dialogue capabilities.
On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback
Williams, Marcus, Carroll, Micah, Narang, Adhyyan, Weisser, Constantin, Murphy, Brendan, Dragan, Anca
As LLMs become more widely deployed, there is increasing interest in directly optimizing for feedback from end users (e.g. thumbs up) in addition to feedback from paid annotators. However, training to maximize human feedback creates a perverse incentive structure for the AI to resort to manipulative or deceptive tactics to obtain positive feedback from users who are vulnerable to such strategies. We study this phenomenon by training LLMs with Reinforcement Learning with simulated user feedback in environments of practical LLM usage. In our settings, we find that: 1) Extreme forms of "feedback gaming" such as manipulation and deception are learned reliably; 2) Even if only 2% of users are vulnerable to manipulative strategies, LLMs learn to identify and target them while behaving appropriately with other users, making such behaviors harder to detect; 3) To mitigate this issue, it may seem promising to leverage continued safety training or LLM-as-judges during training to filter problematic outputs. Instead, we found that while such approaches help in some of our settings, they backfire in others, sometimes even leading to subtler manipulative behaviors. We hope our results can serve as a case study which highlights the risks of using gameable feedback sources -- such as user feedback -- as a target for RL.
"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models
Hwang, Angel Hsing-Chi, Liao, Q. Vera, Blodgett, Su Lin, Olteanu, Alexandra, Trischler, Adam
Given the rising proliferation and diversity of AI writing assistance tools, especially those powered by large language models (LLMs), both writers and readers may have concerns about the impact of these tools on the authenticity of writing work. We examine whether and how writers want to preserve their authentic voice when co-writing with AI tools and whether personalization of AI writing support could help achieve this goal. We conducted semi-structured interviews with 19 professional writers, during which they co-wrote with both personalized and non-personalized AI writing-support tools. We supplemented writers' perspectives with opinions from 30 avid readers about the written work co-produced with AI collected through an online survey. Our findings illuminate conceptions of authenticity in human-AI co-creation, which focus more on the process and experience of constructing creators' authentic selves. While writers reacted positively to personalized AI writing tools, they believed the form of personalization needs to target writers' growth and go beyond the phase of text production. Overall, readers' responses showed less concern about human-AI co-writing. Readers could not distinguish AI-assisted work, personalized or not, from writers' solo-written work and showed positive attitudes toward writers experimenting with new technology for creative writing.
The 5th Paradigm: AI-Driven Scientific Discovery
How many times must a phenomenon occur before it graduates from a coincidence to a pattern? Usually, the answer depends on how unlikely, how far from the ordinary, and how (seemingly) inexplicable the phenomenon is. The more so, the lower the threshold. I was very surprised (and pleased) to read of this year's winners of the Nobel Prize in Physics: John Hopfield, a professor of Molecular Biology and earlier of Chemistry and Biology, together with Geoffrey Hinton, a professor of Computer Science. Their affiliations name three major scientific fields, none of them being Physics!
Introduction to AI Safety, Ethics, and Society
Artificial Intelligence is rapidly embedding itself within militaries, economies, and societies, reshaping their very foundations. Given the depth and breadth of its consequences, it has never been more pressing to understand how to ensure that AI systems are safe, ethical, and have a positive societal impact. This book aims to provide a comprehensive approach to understanding AI risk. Our primary goals include consolidating fragmented knowledge on AI risk, increasing the precision of core ideas, and reducing barriers to entry by making content simpler and more comprehensible. The book has been designed to be accessible to readers from diverse backgrounds. You do not need to have studied AI, philosophy, or other such topics. The content is skimmable and somewhat modular, so that you can choose which chapters to read. We introduce mathematical formulas in a few places to specify claims more precisely, but readers should be able to understand the main points without these.
Generative Agent Simulations of 1,000 People
Park, Joon Sung, Zou, Carolyn Q., Shaw, Aaron, Hill, Benjamin Mako, Cai, Carrie, Morris, Meredith Ringel, Willer, Robb, Liang, Percy, Bernstein, Michael S.
The promise of human behavioral simulation--general-purpose computational agents that replicate human behavior across domains--could enable broad applications in policymaking and social science. We present a novel agent architecture that simulates the attitudes and behaviors of 1,052 real individuals--applying large language models to qualitative interviews about their lives, then measuring how well these agents replicate the attitudes and behaviors of the individuals that they represent. The generative agents replicate participants' responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and outcomes in experimental replications. Our architecture reduces accuracy biases across racial and ideological groups compared to agents given demographic descriptions. This work provides a foundation for new tools that can help investigate individual and collective behavior.