Goto

Collaborating Authors

 improvisation


Emovectors: assessing emotional content in jazz improvisations for creativity evaluation

Jordanous, Anna

arXiv.org Artificial Intelligence

Music improvisation is fascinating to study, being essentially a live demonstration of a creative process. In jazz, musicians often improvise across predefined chord progressions (leadsheets). How do we assess the creativity of jazz improvisations? And can we capture this in automated metrics for creativity for current LLM-based generative systems? Demonstration of emotional involvement is closely linked with creativity in improvisation. Analysing musical audio, can we detect emotional involvement? This study hypothesises that if an improvisation contains more evidence of emotion-laden content, it is more likely to be recognised as creative. An embeddings-based method is proposed for capturing the emotional content in musical improvisations, using a psychologically-grounded classification of musical characteristics associated with emotions. Resulting 'emovectors' are analysed to test the above hypothesis, comparing across multiple improvisations. Capturing emotional content in this quantifiable way can contribute towards new metrics for creativity evaluation that can be applied at scale.


On Improvisation and Open-Endedness: Insights for Experiential AI

Hu, Botao 'Amber'

arXiv.org Artificial Intelligence

Improvisation--the art of spontaneous creation that unfolds moment-to-moment without a scripted outcome--requires practitioners to continuously sense, adapt, and create anew. It is a fundamental mode of human creativity spanning music, dance, and everyday life. The open-ended nature of improvisation produces a stream of novel, unrepeatable moments--an aspect highly valued in artistic creativity. In parallel, open-endedness (OE)--a system's capacity for unbounded novelty and endless "interestingness"--is exemplified in natural or cultural evolution and has been considered "the last grand challenge" in artificial life (ALife). The rise of generative AI now raises the question in computational creativity (CC) research: What makes a "good" improvisation for AI? Can AI learn to improvise in a genuinely open-ended way? In this work-in-progress paper, we report insights from in-depth interviews with 6 experts in improvisation across dance, music, and contact improvisation. We draw systemic connections between human improvisa-tional arts and the design of future experiential AI agents that could improvise alone or alongside humans--or even with other AI agents--embodying qualities of improvisation drawn from practice: active listening (umwelt and awareness), being in the time (mindfulness and ephemerality), embracing the unknown (source of randomness and serendipity), non-judgmental flow (acceptance and dynamical stability, balancing structure and surprise (unpredictable criticality at edge of chaos), imaginative metaphor (synaesthesia and planning), empathy, trust, boundary, and care (mutual theory of mind), and playfulness and intrinsic motivation (maintaining interestingness).


Symbolically Scaffolded Play: Designing Role-Sensitive Prompts for Generative NPC Dialogue

Figueiredo, Vanessa, Elumeze, David

arXiv.org Artificial Intelligence

Large Language Models (LLMs) promise to transform interactive games by enabling non-player characters (NPCs) to sustain unscripted dialogue. Yet it remains unclear whether constrained prompts actually improve player experience. We investigate this question through The Interview, a voice-based detective game powered by GPT-4o. A within-subjects usability study ($N=10$) compared high-constraint (HCP) and low-constraint (LCP) prompts, revealing no reliable experiential differences beyond sensitivity to technical breakdowns. Guided by these findings, we redesigned the HCP into a hybrid JSON+RAG scaffold and conducted a synthetic evaluation with an LLM judge, positioned as an early-stage complement to usability testing. Results uncovered a novel pattern: scaffolding effects were role-dependent: the Interviewer (quest-giver NPC) gained stability, while suspect NPCs lost improvisational believability. These findings overturn the assumption that tighter constraints inherently enhance play. Extending fuzzy-symbolic scaffolding, we introduce \textit{Symbolically Scaffolded Play}, a framework in which symbolic structures are expressed as fuzzy, numerical boundaries that stabilize coherence where needed while preserving improvisation where surprise sustains engagement.


(R)evolution of Programming: Vibe Coding as a Post-Coding Paradigm

Krings, Kevin, Bohn, Nino S., Ludwig, Thomas

arXiv.org Artificial Intelligence

Recent advancements in generative artificial intelligence (GenAI), particularly large language models, have introduced new possibilities for software development practices. In our paper we investigate the emerging Vibe Coding (VC) paradigm that emphasizes intuitive, affect-driven, and improvisational interactions between developers and AI systems. Building upon the discourse of End-User Development (EUD), we explore how VC diverges from conventional programming approaches such as those supported by tools like GitHub Copilot. Through five semi-structured interview sessions with ten experienced software practitioners, we identify five thematic dimensions: creativity, sustainability, the future of programming, collaboration, and criticism. Our analysis conceptualizes VC within the metaphor of co-drifting, contrasting it with the prevalent co-piloting perspective of AI-assisted development. We argue that VC reconfigures the developers role, blurring boundaries between professional and non-developers. While VC enables novel forms of expression and rapid prototyping, it also introduces challenges regarding reproducibility, scalability, and inclusivity. We propose that VC represents a meaningful shift in programming culture, warranting further investigation within human-computer interaction (HCI) and software engineering research.


Detecting and Characterizing Planning in Language Models

Nainani, Jatin, Vaidyanathan, Sankaran, Watts, Connor, Assis, Andre N., Rigg, Alice

arXiv.org Artificial Intelligence

Modern large language models (LLMs) have demonstrated impressive performance across a wide range of multi-step reasoning tasks. Recent work suggests that LLMs may perform planning - selecting a future target token in advance and generating intermediate tokens that lead towards it - rather than merely improvising one token at a time. However, existing studies assume fixed planning horizons and often focus on single prompts or narrow domains. To distinguish planning from improvisation across models and tasks, we present formal and causally grounded criteria for detecting planning and operationalize them as a semi-automated annotation pipeline. We apply this pipeline to both base and instruction-tuned Gemma-2-2B models on the MBPP code generation benchmark and a poem generation task where Claude 3.5 Haiku was previously shown to plan. Our findings show that planning is not universal: unlike Haiku, Gemma-2-2B solves the same poem generation task through improvisation, and on MBPP it switches between planning and improvisation across similar tasks and even successive token predictions. We further show that instruction tuning refines existing planning behaviors in the base model rather than creating them from scratch. Together, these studies provide a reproducible and scalable foundation for mechanistic studies of planning in LLMs.


Theatre in the Loop: A Rehearsal-Based, Collaborative Workflow for Expressive Robotic Behaviours

Panagiotidis, Pavlos, Ngo, Victor Zhi Heung, Myatt, Sean, Patel, Roma, Ramchurn, Rachel, Chamberlain, Alan, Kucukyilmaz, Ayse

arXiv.org Artificial Intelligence

In this paper, we propose theatre-in-the-loop, a framework for developing expressive robot behaviours tailored to artistic performance through a director-guided puppeteering workflow. Leveraging theatrical methods, we use narrative objectives to direct a puppeteer in generating improvised robotic gestures that convey specific emotions. These improvisations are captured and curated to build a dataset of reusable movement templates for standalone playback in future autonomous performances. Initial trials demonstrate the feasibility of this approach, illustrating how the workflow enables precise sculpting of robotic gestures into coherent emotional arcs while revealing challenges posed by the robot's mechanical constraints. We argue that this practice-led framework provides a model for interdisciplinary teams creating socially expressive robot behaviours, contributing to (1) theatre as an interactive training ground for human-robot interaction and (2) co-creation methodologies between humans and machines.


Music Interpretation and Emotion Perception: A Computational and Neurophysiological Investigation

Lyberatos, Vassilis, Kantarelis, Spyridon, Zioga, Ioanna, Anagnostopoulou, Christina, Stamou, Giorgos, Georgaki, Anastasia

arXiv.org Artificial Intelligence

These authors contributed equally to this work. ABSTRACT This study investigates emotional expression and perception in music performance using computational and neurophysiological methods. The influence of different performance settings, such as repertoire, diatonic modal etudes, and improvisation, as well as levels of expressiveness, on performers' emotional communication and listeners' reactions is explored. Professional musicians performed various tasks, and emotional annotations were provided by both performers and the audience. Audio analysis revealed that expressive and improvisational performances exhibited unique acoustic features, while emotion analysis showed stronger emotional responses. Neurophysiological measurements indicated greater relaxation in improvisa-tional performances. This multimodal study highlights the significance of expressivity in enhancing emotional communication and audience engagement. 1. INTRODUCTION In recent years, the study of music performance has become a prominent area of research. While traditional analysis of music often relied on the score, modern research highlights the importance of performance-specific features that distinguish one rendition from another.


A Constructed Response: Designing and Choreographing Robot Arm Movements in Collaborative Dance Improvisation

Chang, Xiaoyu, Zhang, Fan, Fu, Kexue, Diana, Carla, Ju, Wendy, LC, Ray

arXiv.org Artificial Intelligence

Dancers often prototype movements themselves or with each other during improvisation and choreography. How are these interactions altered when physically manipulable technologies are introduced into the creative process? To understand how dancers design and improvise movements while working with instruments capable of non-humanoid movements, we engaged dancers in workshops to co-create movements with a robot arm in one-human-to-one-robot and three-human-to-one-robot settings. We found that dancers produced more fluid movements in one-to-one scenarios, experiencing a stronger sense of connection and presence with the robot as a co-dancer. In three-to-one scenarios, the dancers divided their attention between the human dancers and the robot, resulting in increased perceived use of space and more stop-and-go movements, perceiving the robot as part of the stage background. This work highlights how technologies can drive creativity in movement artists adapting to new ways of working with physical instruments, contributing design insights supporting artistic collaborations with non-humanoid agents.


Next Token Prediction Is a Dead End for Creativity

Olatunji, Ibukun, Sheppard, Mark

arXiv.org Artificial Intelligence

This position paper argues that token prediction is fundamentally misaligned with real creativity. While next-token models have enabled impressive advances in language generation, their architecture favours surface-level coherence over spontaneity, originality, and improvisational risk. In contrast, creative acts, particularly in live performance domains, require dynamic responsiveness and stylistic divergence, enabling humans to transcend pre-learned patterns in the moment. We use battle rap as a case study to expose the limitations of predictive systems, demonstrating that they cannot truly engage in adversarial or emotionally resonant exchanges. As a result, such models fail to support the interactive flow states where human creators "lose themselves in the moment." Rather than pursuing greater predictive accuracy, we argue that AI research should embrace dialogue as a form of co-negotiated creative agency. This shift calls for approaches that prioritize real-time interaction, rhythmic alignment, and adaptive generative control. By reframing creativity as an interactive process rather than a predictive output, we offer a vision for AI systems that are more expressive, responsive, and aligned with human creative practice.


Somatic Safety: An Embodied Approach Towards Safe Human-Robot Interaction

Benford, Steve, Schneiders, Eike, Avila, Juan Pablo Martinez, Caleb-Solly, Praminda, Brundell, Patrick Robert, Castle-Green, Simon, Zhou, Feng, Garrett, Rachael, Höök, Kristina, Whatley, Sarah, Marsh, Kate, Tennent, Paul

arXiv.org Artificial Intelligence

As robots enter the messy human world so the vital matter of safety takes on a fresh complexion with physical contact becoming inevitable and even desirable. We report on an artistic-exploration of how dancers, working as part of a multidisciplinary team, engaged in contact improvisation exercises to explore the opportunities and challenges of dancing with cobots. We reveal how they employed their honed bodily senses and physical skills to engage with the robots aesthetically and yet safely, interleaving improvised physical manipulations with reflections to grow their knowledge of how the robots behaved and felt. We introduce somatic safety, a holistic mind-body approach in which safety is learned, felt and enacted through bodily contact with robots in addition to being reasoned about. We conclude that robots need to be better designed for people to hold them and might recognise tacit safety cues among people.We propose that safety should be learned through iterative bodily experience interleaved with reflection.