Media
Sports Commentary Recommendation System (SCoReS): Machine Learning for Automated Narrative
Lee, Greg Michael (University of Alberta) | Bulitko, Vadim (University of Alberta) | Ludvig, Elliot (Princeton University)
Automated sports commentary is a form of automated narrative. Sports commentary exists to keep the viewer informed and entertained. One way to entertain the viewer is by telling brief stories relevant to the game in progress. We introduce a system called the Sports Commentary Recommendation System (SCoReS) that can automatically suggest stories for commentators to tell during games. Through several user studies, we compared commentary using SCoReS to three other types of commentary and show that SCoReS adds significantly to the broadcast across several enjoyment metrics. We also collected interview data from professional sports commentators who positively evaluated a demonstration of the system. We conclude that SCoReS can be a useful broadcast tool, effective at selecting stories that add to the enjoyment and watchability of sports. SCoReS is a step toward automating sports commentary and, thus, automating narrative.
Embracing the Bias of the Machine: Exploring Non-Human Fitness Functions
Eigenfeldt, Arne (Simon Fraser University)
Autonomous aesthetic evaluation is the Holy Grail of generative music, and one of the great challenges of computational creativity. Unlike most other computational activities, there is no notion of optimality in evaluating creative output: there are subjective impressions involved, and framing obviously plays a big role. When developing metacreative systems, a purely objective fitness function is not available: the designer is thus faced with how much of their own aesthetic to include. Can a generative system be free of the designer’s bias? This paper presents a system that incorporates an aesthetic selection process that allows for both human-designed and non-human fitness functions.
Aesthetic Considerations for Automated Platformer Design
Cook, Michael (Imperial College, London) | Colton, Simon (Imperial College, London ) | Pease, Alison (Imperial College, London)
We describe ANGELINA3, a system that can automatically develop games along a defined theme, by selecting appropriate multimedia content from a variety of sources and incorporating it into a game's design. We discuss these capabilities in the context of the FACE model for assessing progress in the building of creative systems, and discuss how ANGELINA3 can be improved through further work.
Demo: A Computer-Assisted Approach to Composing with MaestroGenesis
Szerlip, Paul A. (University of Central Florida) | Hoover, Amy K. (University of Central Florida) | Stanley, Kenneth O. (University of Central Florida)
This demonstration presents MaestroGenesis, a program that helps users create complete polyphonic musical pieces from as little as a simple, human composed monophonic melody. MaestroGenesis creates music by exploiting two key ideas behind the functional scaffolding for musical composition (FSMC) approach: (1) that music a function of time and (2) that functional transformations of initial human starting melodies, or scaffolds, inherit some of the essential human qualities contained in the scaffold. Music in FSMC is represented as a functional relationship between the scaffold and a generated accompaniment. The GUI helps users evolve these functions by importing and developing their music through a breeding process akin to animal breeding, called interactive evolutionary computation. Some resulting pieces are indistinguishable from completely human-composed pieces.
Toward a Narrative Comprehension Model of Cinematic Generation for 3D Virtual Environments
Cassell, Bradley Alan (North Carolina State University)
Most systems for generating cinematic shot sequences for virtual environments focus on the low-level problems of camera placement. While this approach will create a sequence of camera shots which film individual events in a virtual environment, it does not account for the high-level effects shot sequences have on viewer inferences. There are systems which are based on well known cinematography principles such as the rule of thirds and other framing principals, however these usually utilize schemas or predefined shots and do not reason about the high level cognitive effects on the viewer. In this paper a system is proposed which can reason directly about these high-level cognitive and narrative effects of a shot sequence on the viewer’s mental state.
‘Xa-lan’: Algorithmic Generation of Expressive Music Scores Based on Signal Analysis and Graphical Transformations
Rodriguez, Mauricio E. (Stanford University)
Xa-lan is a computer program written in Common-LISP to generate music scores with a high level of notational/symbolic expressivity. Generation is driven by audio-analysis of melodic profiles. Once a melodic contour is input to the software, graphic transformations of the original profile stochastically control the different notational elements of the score. The Xa-lan routines display their final output using the ‘Expressive Notation Package’ of PWGL, a LISP-based visual composition environment. A full range of traditional and non-conventional music notation elements can be algorithmically generated with Xa-lan, retrieving to the user a ‘ready-to-play’ or fully ex-pressive music score.
Music Design with Audio Oracle Using Information Rate
Dubnov, Shlomo (University of California, San Diego) | Assayag, Gerard (Ircam)
In the proposed demo we will present a method for design of musical composition using the Audio Oracle (AO) - a machine improvisation method that constructs and produces variation from a music recording using a graph of repeated factors found in the recording. The compositional / improvisation use of AO involves controlling the rate of recombination, the length of common history (length of the repeated factors) and selection of regions in the AO where the machine operates. One of the challenges in working with AO is understanding the generative potential of different audio materials and marking and allocating regions in a recording where AO should operate to achieve the desired musical result. Recently we introduced a novel analysis method "on top" of the AO structure that captures aspects of order and complexity that we call Information Rate. This measure, belonging broadly to a field of study know as Musical Information Dynamics, is related to Bense formulation of aesthetics in terms of entropy and compression. In the demo we will briefly explain the theory behind Audio Oracle and Information Rate and demonstrate a process of designing a composition / structured improvisation based on analysis of the AO graph. In contrast to the usual live interaction method where both the audio input to the oracle and the improvisation are done "on the fly", in this presentation the audio analysis will be done prior to the presentation, while during the talk we will show the process of planning a composition and then do a live performance with the AO based on this design.
Algorithmically Flexible Style Composition Through Multi-Objective Fitness Functions
Murray, Skyler (Brigham Young University) | Ventura, Dan (Brigham Young University)
Creating a musical fitness function is largely subjective and can be critically affected by the designer's biases. Previous attempts to create such functions for use in genetic algorithms lack scope or are prejudiced to a certain genre of music. They also are limited to producing music strictly in the style determined by the programmer. We show in this paper that musical feature extractors, which avoid the challenges of qualitative judgment, enable creation of a multi-objective function for direct music production. The main result is that the multi-objective fitness function enables creation of music with varying identifiable styles. To demonstrate this, we use three different multi-objective fitness functions to create three distinct sets of musical melodies. We then evaluate the distinctness of these sets using three different approaches: a set of traditional computational clustering metrics; a survey of non-musicians; and analysis by three trained musicians.
Autonomy in Music-Generating Systems
Bown, Oliver Roland (University of Sydney) | Martin, Aengus (University of Sydney)
The word autonomy is often used in the discussion of software-based music-generating systems. Whilst the term conveys a very clear concept — the sense of self-determination of a system — attempts to formalise autonomy are at an early stage, and the term is subject to a range of interpretations when practically applied. We consider how the evaluation of music-generating systems will be enhanced by a clearer understanding of autonomy and its application to music. We discuss existing definitions and approaches to quantifying autonomy and consider, through a series of examples, the information that is required in order to make precise formal judgements about autonomy, and the identification of relevant levels at which the principle of autonomy applies in music. We conclude that automated measures can supplement human evaluation of autonomy, but that (a) automated measures must be supported by sound reasoning about the features and timescales used in the measurement, and (b) they are improved by a having knowledge of the internal working of the system, rather than taking a black box approach. We consider multi-dimensional representations of system behaviour that may capture a richer sense of the notion of autonomy. Finally, we propose an approach to automatically probing music systems as a means of determining an autonomy `portrait'.
The Melody Triangle: Exploring Pattern and Predictability in Music
Ekeus, Henrik (Queen Mary University of London) | Abdallah, Samer (Queen Mary University of London) | Plumbley, Mark (Queen Mary University of London) | McOwan, Peter (Queen Mary University of London)
The Melody Triangle is an interface for the discovery of melodic materials, where the input – positions within a triangle – directly map to information theoretic properties of the output. A model of human expectation and surprise in the perception of music, information dynamics, is used to ‘map out’ a musical generative system’s parameter space. This enables a user to explore the possibilities afforded by a generative algorithm, in this case Markov chains, not by directly selecting parameters, but by specifying the subjective predictability of the output sequence. We describe some of the relevant ideas from information dynamics and how the Melody Triangle is defined in terms of these. We describe its incarnation as a screen based performance tool and compositional aid for the generation of musical textures; the users control at the abstract level of randomness and predictability, and some pilot studies carried out with it. We also briefly outline a multi-user installation, where collabo- ration in a performative setting provides a playful yet informative way to explore expectation and surprise in music, and a forthcoming mobile phone version of the Melody Triangle.