The recent rise in capabilities of AI-based music generation tools has created an upheaval in the music industry, necessitating the creation of accurate methods to detect such AI-generated content. This can be done using audio-based detectors; however, it has been shown that they struggle to generalize to unseen generators or when the audio is perturbed. Furthermore, recent work used accurate and cleanly formatted lyrics sourced from a lyrics provider database to detect AI-generated music. However, in practice, such perfect lyrics are not available (only the audio is); this leaves a substantial gap in applicability in real-life use cases. In this work, we instead propose solving this gap by transcribing songs using general automatic speech recognition (ASR) models. We do this using several detectors. The results on diverse, multi-genre, and multi-lingual lyrics show generally strong detection performance across languages and genres, particularly for our best-performing model using Whisper large-v2 and LLM2Vec embeddings. In addition, we show that our method is more robust than state-of-the-art audio-based ones when the audio is perturbed in different ways and when evaluated on different music generators. Our code is available at https://github.com/deezer/robust-AI-lyrics-detection.

In an age when artificial intelligence (AI) is touted to take over many jobs across various industries, songwriting would be the last thing coming to our minds -- after all, the best songs are about the human experience and how could a robot replicate that? However, a study by ticket marketplace TickPick analyzed whether AI could, in fact, replace songwriters and their findings might not please many. For the study, the AI software was trained just for five hours with lyrics across various genres, including pop, country, hip-hop and rock. Later, the AI-generated lyrics were then tested among 1,003 music fans via Amazon Mechanical Turk "with the intent of testing human ability to correctly identify AI-generated lyrics, as well as their opinion on creativity, emotionality and favorability of various lyrics presented to them." Models were trained between five and 12 hours after lyrics were taken from genius.com and grouped by genre.