Losses, Dissonances, and Distortions

In recent years, there has been a growing interest in using machine learning models for creative purposes. In most cases, this is with the use of large generative models which, as their name implies, can generate high-quality and realistic outputs in music [Huang et al., 2019], images [Esser et al., 2021], text [Brown et al., 2020], and others. The standard approach for artistic creation using these models is to take a pre-trained model (or set of models) and use them for producing output. The artist directs the model's generation by "navigating" the latent space [Castro, 2020], fine-tuning the trained parameters [Dinculescu et al., 2019], or using the model's output to steer another generative process [White, 2019, Castro, 2019]. At a high-level what all these approaches are doing is converting the numerical signal of a machine learning model's output into art, whether implicitly or explicitly. However, in most (if not all) cases they only do so after the initial model has been trained.