Hayes, Ben
AI (r)evolution -- where are we heading? Thoughts about the future of music and sound technologies in the era of deep learning
Bindi, Giovanni, Demerlé, Nils, Diaz, Rodrigo, Genova, David, Golvet, Aliénor, Hayes, Ben, Huang, Jiawen, Liu, Lele, Martos, Vincent, Nabi, Sarah, Pelinski, Teresa, Renault, Lenny, Sarkar, Saurjya, Sarmento, Pedro, Vahidi, Cyrus, Wolstanholme, Lewis, Zhang, Yixiao, Roebel, Axel, Bryan-Kinns, Nick, Giavitto, Jean-Louis, Barthet, Mathieu
Artificial Intelligence (AI) technologies such as deep learning are evolving very quickly bringing many changes to our everyday lives. To explore the future impact and potential of AI in the field of music and sound technologies a doctoral day was held between Queen Mary University of London (QMUL, UK) and Sciences et Technologies de la Musique et du Son (STMS, France). Prompt questions about current trends in AI and music were generated by academics from QMUL and STMS. Students from the two institutions then debated these questions. This report presents a summary of the student debates on the topics of: Data, Impact, and the Environment; Responsible Innovation and Creative Practice; Creativity and Bias; and From Tools to the Singularity. The students represent the future generation of AI and music researchers. The academics represent the incumbent establishment. The student debates reported here capture visions, dreams, concerns, uncertainties, and contentious issues for the future of AI and music as the establishment is rightfully challenged by the next generation.
The Responsibility Problem in Neural Networks with Unordered Targets
Hayes, Ben, Saitis, Charalampos, Fazekas, György
We discuss the discontinuities that arise when mapping unordered objects to neural network outputs of fixed permutation, referred to as the responsibility problem. Prior work has proved the existence of the issue by identifying a single discontinuity. Here, we show that discontinuities under such models are uncountably infinite, motivating further research into neural networks for unordered data. The responsibility problem (Zhang et al., 2020b) describes an issue when training neural networks with unordered targets: the fixed permutation of output units requires that each assume a "responsibility" for some element. For feed-forward networks, the worst-case approximation of such discontinuous functions is arbitrarily poor for at least some subset of the input space (Kratsios & Zamanlooy, 2022) Empirically, degraded performance has been observed on set prediction tasks (Zhang et al., 2020a), motivating research into architectures for set generation which circumvent these discontinuities (Zhang et al., 2020a; Kosiorek et al., 2020; Rezatofighi et al., 2018).
Rigid-Body Sound Synthesis with Differentiable Modal Resonators
Diaz, Rodrigo, Hayes, Ben, Saitis, Charalampos, Fazekas, György, Sandler, Mark
Physical models of rigid bodies are used for sound synthesis in applications from virtual environments to music production. Traditional methods such as modal synthesis often rely on computationally expensive numerical solvers, while recent deep learning approaches are limited by post-processing of their results. In this work we present a novel end-to-end framework for training a deep neural network to generate modal resonators for a given 2D shape and material, using a bank of differentiable IIR filters. We demonstrate our method on a dataset of synthetic objects, but train our model using an audio-domain objective, paving the way for physically-informed synthesisers to be learned directly from recordings of real-world objects.