Media
ChordSync: Conformer-Based Alignment of Chord Annotations to Music Audio
Poltronieri, Andrea, Presutti, Valentina, Rocamora, Martín
In the Western music tradition, chords are the main constituent components of harmony, a fundamental dimension of music. Despite its relevance for several Music Information Retrieval (MIR) tasks, chord-annotated audio datasets are limited and need more diversity. One way to improve those resources is to leverage the large number of chord annotations available online, but this requires aligning them with music audio. However, existing audio-to-score alignment techniques, which typically rely on Dynamic Time Warping (DTW), fail to address this challenge, as they require weakly aligned data for precise synchronisation. In this paper, we introduce ChordSync, a novel conformer-based model designed to seamlessly align chord annotations with audio, eliminating the need for weak alignment. We also provide a pre-trained model and a user-friendly library, enabling users to synchronise chord annotations with audio tracks effortlessly. In this way, ChordSync creates opportunities for harnessing crowd-sourced chord data for MIR, especially in audio chord estimation, thereby facilitating the generation of novel datasets. Additionally, our system extends its utility to music education, enhancing music learning experiences by providing accurately aligned annotations, thus enabling learners to engage in synchronised musical practices.
Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audio
Batlle-Roca, Roser, Liao, Wei-Hisang, Serra, Xavier, Mitsufuji, Yuki, Gómez, Emilia
Recent advancements in music generation are raising multiple concerns about the implications of AI in creative music processes, current business models and impacts related to intellectual property management. A relevant discussion and related technical challenge is the potential replication and plagiarism of the training set in AI-generated music, which could lead to misuse of data and intellectual property rights violations. To tackle this issue, we present the Music Replication Assessment (MiRA) tool: a model-independent open evaluation method based on diverse audio music similarity metrics to assess data replication. We evaluate the ability of five metrics to identify exact replication by conducting a controlled replication experiment in different music genres using synthetic samples. Our results show that the proposed methodology can estimate exact data replication with a proportion higher than 10%. By introducing the MiRA tool, we intend to encourage the open evaluation of music-generative models by researchers, developers, and users concerning data replication, highlighting the importance of the ethical, social, legal, and economic consequences. Code and examples are available for reproducibility purposes.
Video games strike rumbles on in row over AI
Actors from the world of gaming went on strike last week, in a row about the use of artificial intelligence (AI) and the threat it poses to their livelihoods. It has reignited the debate about how the entertainment industry is adapting to new technology. When actor Jennifer Hale talks, you listen. Her delivery is measured and surgically precise, yet her tone has a warmth that most ASMR creators would envy. She could read the phone book and you'd pay attention.
Fox News AI Newsletter: China forces AI to toe party line
Alex Galvagni, CEO of Age of Learning and a former artificial intelligence researcher with NASA, says advances in AI now make it possible to deliver to children "a personalized and supportive" experience in education. A man walks past a photo of Chinese President Xi Jinping at the Museum of the Communist Party of China in Beijing on March 3, 2023, ahead of the opening of the annual session of the National Peoples Congress on March 5. (GREG BAKER/AFP via Getty Images) BRAINWASHED: China has once again extended its policy of censorship and surveillance as it looks to keep artificial intelligence models in check even as it races to advance the ever-expanding technology. 'TIDE OF DISINFORMATION': The Federal Communications Commission announced a new proposed regulation that would require the use of artificial intelligence in broadcast TV and radio ads to be disclosed to the audience through an on-air announcement. THREATENED: With the rise of artificial intelligence technology in the form of deepfakes, as well as misinformation campaigns that can manipulate peoples' emotions, don't let yourself fall for them. These three categories of cyberthreats should be on the radar ahead of elections.
Review of Explainable Graph-Based Recommender Systems
Markchom, Thanet, Liang, Huizhi, Ferryman, James
Explainability of recommender systems has become essential to ensure users' trust and satisfaction. Various types of explainable recommender systems have been proposed including explainable graph-based recommender systems. This review paper discusses state-of-the-art approaches of these systems and categorizes them based on three aspects: learning methods, explaining methods, and explanation types. It also explores the commonly used datasets, explainability evaluation methods, and future directions of this research area. Compared with the existing review papers, this paper focuses on explainability based on graphs and covers the topics required for developing novel explainable graph-based recommender systems.
Characterizing User Archetypes and Discussions on Scored.co
Failla, Andrea, Citraro, Salvatore, Rossetti, Giulio, Cauteruccio, Francesco
In recent years, the proliferation of social platforms has drastically transformed the way individuals interact, organize, and share information. In this scenario, we experience an unprecedented increase in the scale and complexity of interactions and, at the same time, little to no research about some fringe social platforms. In this paper, we present a multi-dimensional framework for characterizing nodes and hyperedges in social hypernetworks, with a focus on the understudied alt-right platform Scored.co. Our approach integrates the possibility of studying higher-order interactions, thanks to the hypernetwork representation, and various node features such as user activity, sentiment, and toxicity, with the aim to define distinct user archetypes and understand their roles within the network. Utilizing a comprehensive dataset from Scored.co, we analyze the dynamics of these archetypes over time and explore their interactions and influence within the community. The framework's versatility allows for detailed analysis of both individual user behaviors and broader social structures. Our findings highlight the importance of higher-order interactions in understanding social dynamics, offering new insights into the roles and behaviors that emerge in complex online environments.
Beat this! Accurate beat tracking without DBN postprocessing
Foscarin, Francesco, Schlüter, Jan, Widmer, Gerhard
We propose a system for tracking beats and downbeats with two objectives: generality across a diverse music range, and high accuracy. We achieve generality by training on multiple datasets -- including solo instrument recordings, pieces with time signature changes, and classical music with high tempo variations -- and by removing the commonly used Dynamic Bayesian Network (DBN) postprocessing, which introduces constraints on the meter and tempo. For high accuracy, among other improvements, we develop a loss function tolerant to small time shifts of annotations, and an architecture alternating convolutions with transformers either over frequency or time. Our system surpasses the current state of the art in F1 score despite using no DBN. However, it can still fail, especially for difficult and underrepresented genres, and performs worse on continuity metrics, so we publish our model, code, and preprocessed datasets, and invite others to beat this.
Combining audio control and style transfer using latent diffusion
Demerlé, Nils, Esling, Philippe, Doras, Guillaume, Genova, David
Deep generative models are now able to synthesize high-quality audio signals, shifting the critical aspect in their development from audio quality to control capabilities. Although text-to-music generation is getting largely adopted by the general public, explicit control and example-based style transfer are more adequate modalities to capture the intents of artists and musicians. In this paper, we aim to unify explicit control and style transfer within a single model by separating local and global information to capture musical structure and timbre respectively. To do so, we leverage the capabilities of diffusion autoencoders to extract semantic features, in order to build two representation spaces. We enforce disentanglement between those spaces using an adversarial criterion and a two-stage training strategy. Our resulting model can generate audio matching a timbre target, while specifying structure either with explicit controls or through another audio example. We evaluate our model on one-shot timbre transfer and MIDI-to-audio tasks on instrumental recordings and show that we outperform existing baselines in terms of audio quality and target fidelity. Furthermore, we show that our method can generate cover versions of complete musical pieces by transferring rhythmic and melodic content to the style of a target audio in a different genre.
Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs
Markowitz, Elan, Ramakrishna, Anil, Dhamala, Jwala, Mehrabi, Ninareh, Peris, Charith, Gupta, Rahul, Chang, Kai-Wei, Galstyan, Aram
Knowledge graphs (KGs) complement Large Language Models (LLMs) by providing reliable, structured, domain-specific, and up-to-date external knowledge. However, KGs and LLMs are often developed separately and must be integrated after training. We introduce Tree-of-Traversals, a novel zero-shot reasoning algorithm that enables augmentation of black-box LLMs with one or more KGs. The algorithm equips a LLM with actions for interfacing a KG and enables the LLM to perform tree search over possible thoughts and actions to find high confidence reasoning paths. We evaluate on two popular benchmark datasets. Our results show that Tree-of-Traversals significantly improves performance on question answering and KG question answering tasks. Code is available at \url{https://github.com/amazon-science/tree-of-traversals}
Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models
Wu, Zhengxuan, Zhang, Yuhao, Qi, Peng, Xu, Yumo, Han, Rujun, Zhang, Yian, Chen, Jifan, Min, Bonan, Huang, Zhiheng
Modern language models (LMs) need to follow human instructions while being faithful; yet, they often fail to achieve both. Here, we provide concrete evidence of a trade-off between instruction following (i.e., follow open-ended instructions) and faithfulness (i.e., ground responses in given context) when training LMs with these objectives. For instance, fine-tuning LLaMA-7B on instruction following datasets renders it less faithful. Conversely, instruction-tuned Vicuna-7B shows degraded performance at following instructions when further optimized on tasks that require contextual grounding. One common remedy is multi-task learning (MTL) with data mixing, yet it remains far from achieving a synergic outcome. We propose a simple yet effective method that relies on Rejection Sampling for Continued Self-instruction Tuning (ReSet), which significantly outperforms vanilla MTL. Surprisingly, we find that less is more, as training ReSet with high-quality, yet substantially smaller data (three-fold less) yields superior results. Our findings offer a better understanding of objective discrepancies in alignment training of LMs.