Goto

Collaborating Authors

 Media


Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation

arXiv.org Artificial Intelligence

Symbolic Music, akin to language, can be encoded in discrete symbols. Recent research has extended the application of large language models (LLMs) such as GPT-4 and Llama2 to the symbolic music domain including understanding and generation. Yet scant research explores the details of how these LLMs perform on advanced music understanding and conditioned generation, especially from the multi-step reasoning perspective, which is a critical aspect in the conditioned, editable, and interactive human-computer co-creation process. This study conducts a thorough investigation of LLMs' capability and limitations in symbolic music processing. We identify that current LLMs exhibit poor performance in song-level multi-step music reasoning, and typically fail to leverage learned music knowledge when addressing complex musical tasks. An analysis of LLMs' responses highlights distinctly their pros and cons. Our findings suggest achieving advanced musical capability is not intrinsically obtained by LLMs, and future research should focus more on bridging the gap between music knowledge and reasoning, to improve the co-creation experience for musicians.


Between the AI and Me: Analysing Listeners' Perspectives on AI- and Human-Composed Progressive Metal Music

arXiv.org Artificial Intelligence

Generative AI models have recently blossomed, significantly impacting artistic and musical traditions. Research investigating how humans interact with and deem these models is therefore crucial. Through a listening and reflection study, we explore participants' perspectives on AI- vs human-generated progressive metal, in symbolic format, using rock music as a control group. AI-generated examples were produced by ProgGP, a Transformer-based model. We propose a mixed methods approach to assess the effects of generation type (human vs. AI), genre (progressive metal vs. rock), and curation process (random vs. cherry-picked). This combines quantitative feedback on genre congruence, preference, creativity, consistency, playability, humanness, and repeatability, and qualitative feedback to provide insights into listeners' experiences. A total of 32 progressive metal fans completed the study. Our findings validate the use of fine-tuning to achieve genre-specific specialization in AI music generation, as listeners could distinguish between AI-generated rock and progressive metal. Despite some AI-generated excerpts receiving similar ratings to human music, listeners exhibited a preference for human compositions. Thematic analysis identified key features for genre and AI vs. human distinctions. Finally, we consider the ethical implications of our work in promoting musical data diversity within MIR research by focusing on an under-explored genre.


Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation

arXiv.org Artificial Intelligence

Advanced Artificial Intelligence (AI) systems, specifically large language models (LLMs), have the capability to generate not just misinformation, but also deceptive explanations that can justify and propagate false information and erode trust in the truth. We examined the impact of deceptive AI generated explanations on individuals' beliefs in a pre-registered online experiment with 23,840 observations from 1,192 participants. We found that in addition to being more persuasive than accurate and honest explanations, AI-generated deceptive explanations can significantly amplify belief in false news headlines and undermine true ones as compared to AI systems that simply classify the headline incorrectly as being true/false. Moreover, our results show that personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from these effects caused by deceptive AI generated explanations. Instead, our results show that the logical validity of AI generated deceptive explanations, that is whether the explanation has a causal effect on the truthfulness of the AI's classification, plays a critical role in countering their persuasiveness - with logically invalid explanations being deemed less credible. This underscores the importance of teaching logical reasoning and critical thinking skills to identify logically invalid arguments, fostering greater resilience against advanced AI-driven misinformation.


Generative Expressive Conversational Speech Synthesis

arXiv.org Artificial Intelligence

Conversational Speech Synthesis (CSS) aims to express a target utterance with the proper speaking style in a user-agent conversation setting. Existing CSS methods employ effective multi-modal context modeling techniques to achieve empathy understanding and expression. However, they often need to design complex network architectures and meticulously optimize the modules within them. In addition, due to the limitations of small-scale datasets containing scripted recording styles, they often fail to simulate real natural conversational styles. To address the above issues, we propose a novel generative expressive CSS system, termed GPT-Talker.We transform the multimodal information of the multi-turn dialogue history into discrete token sequences and seamlessly integrate them to form a comprehensive user-agent dialogue context. Leveraging the power of GPT, we predict the token sequence, that includes both semantic and style knowledge, of response for the agent. After that, the expressive conversational speech is synthesized by the conversation-enriched VITS to deliver feedback to the user.Furthermore, we propose a large-scale Natural CSS Dataset called NCSSD, that includes both naturally recorded conversational speech in improvised styles and dialogues extracted from TV shows. It encompasses both Chinese and English languages, with a total duration of 236 hours.We conducted comprehensive experiments on the reliability of the NCSSD and the effectiveness of our GPT-Talker. Both subjective and objective evaluations demonstrate that our model outperforms other state-of-the-art CSS systems significantly in terms of naturalness and expressiveness. The Code, Dataset, and Pre-trained Model are available at: https://github.com/AI-S2-Lab/GPT-Talker.


Stable Audio Open

arXiv.org Artificial Intelligence

Open generative models are vitally important for the community, allowing for fine-tunes and serving as baselines when presenting new models. However, most current text-to-audio models are private and not accessible for artists and researchers to build upon. Here we describe the architecture and training process of a new open-weights text-to-audio model trained with Creative Commons data. Our evaluation shows that the model's performance is competitive with the state-of-the-art across various metrics. Notably, the reported FDopenl3 results (measuring the realism of the generations) showcase its potential for high-quality stereo sound synthesis at 44.1kHz.


The AI Search War Has Begun

The Atlantic - Technology

Every second of every day, people across the world type tens of thousands of queries into Google, adding up to trillions of searches a year. Google and a few other search engines are the portal through which several billion people navigate the internet. Many of the world's most powerful tech companies, including Google, Microsoft, and OpenAI, have recently spotted an opportunity to remake that gateway with generative AI, and they are racing to seize it. And as of this week, the generative-AI search wars are in full swing. The value of an AI-powered search bar is straightforward: Instead of having to open and read multiple links, wouldn't it be better to type your query into a chatbot and receive an immediate, comprehensive answer?


The Video Game Industry Is More Successful Than Ever. Why Are Its Workers Treated Like Garbage?

Slate

Video game workers--whatever their job, employer, or status--have clearly had enough. This month alone, the labor movement has made some of its biggest advancements ever in organizing the techies, artists, and creatives who keep the largest, most culturally significant sector of the global entertainment industry running and thriving. First, on July 19, came "wall-to-wall" union approval at Fallout-maker Bethesda Game Studios, which meant that everyone from engineers to artists could establish a comprehensive unit with the Communications Workers of America. They quickly earned recognition from parent company Microsoft, marking the first wall-to-wall effort to succeed at any of the Big Tech firm's gaming studios. On July 24, even more company workers got into the game.


Star Wars Outlaws: what to expect from Ubisoft's galactic adventure

The Guardian

About 10 minutes into the latest preview build of Star Wars Outlaws, Ubisoft's forthcoming open-world adventure, lead character Kay Vess enters Mirogana: a densely populated, worn-down city on the desolate moon of Toshara. Around us is a mix of sandstone hovels and metallic sci-fi buildings, crammed with flickering computer panels, neon signs and holographic adverts. Exotic aliens lurk in quiet corners, R2 droids glide past twittering to themselves. Nearby is a cantina, its shady clientele visible through the smoky doorway, and just to the side is a dimly lit gambling parlour. As you explore, robotic voices read out imperial propaganda over public address systems and stormtroopers patrol the streets, checking IDs. At least as far as this lifelong Star Wars fan is concerned, these moments perfectly capture the aesthetics and atmosphere of the original trilogy.


Perplexity will put ads in its AI search engine and share revenue with publishers

Engadget

When people type a question into Perplexity, the two-year-old search engine scours the internet and uses information from multiple sources, including online publishers, to synthesize an answer using AI. Soon, Perplexity will start sharing revenue with some publishers as part of an advertising platform it plans to launch around the end of September, the company announced on Tuesday. The initiative, known as the Perplexity Publishers' Program, comes less than two months after the San Francisco-based startup backed by investors like Jeff Bezos and NVIDIA, and valued at 3 billion, came under fire from Forbes, Wired, and Condé Nast for allegedly scraping content without permission and ignoring robots.txt, Perplexity's initial partners include TIME, Fortune, The Texas Tribune, Der Spiegel and Automattic, the company behind Wordpress.com. It's not clear exactly how much revenue Perplexity will share with publishers.


Emotion-driven Piano Music Generation via Two-stage Disentanglement and Functional Representation

arXiv.org Artificial Intelligence

Managing the emotional aspect remains a challenge in automatic music generation. Prior works aim to learn various emotions at once, leading to inadequate modeling. This paper explores the disentanglement of emotions in piano performance generation through a two-stage framework. The first stage focuses on valence modeling of lead sheet, and the second stage addresses arousal modeling by introducing performance-level attributes. To further capture features that shape valence, an aspect less explored by previous approaches, we introduce a novel functional representation of symbolic music. This representation aims to capture the emotional impact of major-minor tonality, as well as the interactions among notes, chords, and key signatures. Objective and subjective experiments validate the effectiveness of our framework in both emotional valence and arousal modeling. We further leverage our framework in a novel application of emotional controls, showing a broad potential in emotion-driven music generation.