We describe an unconventional line of attack in our quest to teach machines how to rap battle by improvising hip hop lyrics on the fly, in which a novel recursive bilingual neural network, TRAAM, implicitly learns soft, context-dependent generalizations over the structural relationships between associated parts of challenge and response raps, while avoiding the exponential complexity costs that symbolic models would require. TRAAM learns feature vectors simultaneously using context from both the challenge and the response, such that challenge-response association patterns with similar structure tend to have similar vectors. Improvisation is modeled as a quasi-translation learning problem, where TRAAM is trained to improvise fluent and rhyming responses to challenge lyrics. The soft structural relationships learned by our TRAAM model are used to improve the probabilistic responses generated by our improvisational response component.

While most natural language understanding systems rely on a pipeline-based architecture, certain human text interpretation methods are based on a cyclic process between the whole text and its parts: the hermeneutic circle. In the task of automatically identifying characters and their narrative roles, we propose a feedback-loop-based approach where the output of later modules of the pipeline is fed back to earlier ones. We analyze this approach using a corpus of 21 Russian folktales. Initial results show that feeding back high-level narrative information improves the performance of some NLP tasks.

Aesthetic evaluation of Chinese calligraphy is one of the most challenging tasks in Artificial Intelligence. This paper attempts to solve this problem by proposing a number of aesthetic feature representations and feeding them into Artificial Neural Networks. Specifically, 22 global shape features are presented to describe a given handwritten Chinese character from different aspects according to classical calligraphic rules, and a new 10-dimensional feature vector is introduced to represent the component layout information using sparse coding. Moreover, a Chinese Handwriting Aesthetic Evaluation Database (CHAED) is also built by collecting 1000 Chinese handwriting images with diverse aesthetic qualities and inviting 33 subjects to evaluate the aesthetic quality for each calligraphic image. Finally, back propagation neural networks are constructed with the concatenation of the proposed features as input and then trained on our CHAED database for the aesthetic evaluation of Chinese calligraphy. Experimental results demonstrate that the proposed AI system provides a comparable performance with human evaluation. Through our experiments, we also compare the importance of each individual feature and reveal the relationship between our aesthetic features and the aesthetic perceptions of human beings.

Abstract artists use non-figurative elements (i.e. colours, lines, shapes, and textures) to convey emotions and often rely on the titles of their various compositions to generate (or enhance) an emotional reaction in the audience. Several psychological works observed that the metadata (i.e., titles, description and/or artist statements) associated with paintings increase the understanding and the aesthetic appreciation of artworks. In this paper we explore if the same metadata could facilitate the computational analysis of artworks, and reveal what kind of emotional responses they awake. To this end, we employ computer vision and sentiment analysis to learn statistical patterns associated with positive and negative emotions on abstract paintings. We propose a multimodal approach which combines both visual and metadata features in order to improve the machine performance. In particular, we propose a novel joint flexible Schatten p-norm model which can exploit the sharing patterns between visual and textual information for abstract painting emotion analysis. Moreover, we conduct a qualitative analysis on the cases in which metadata help improving the machine performance.

In this paper we introduce our haiku generator, which, in contrast to other systems, is not restricted to limited classic vocabulary sets and preserves a classic style without becoming too random and abstract because it performs a semantic integrity check using the Internet. Moreover, it is able to analyze blog entry input and, by using nouns and adjectives for web-mining, to stay on topic and still preserve kigo, traditional seasonal words used in Japanese poetry. The haiku generator utilizes grammar templates automatically generated from poems written by Japanese poets and a lexicon of 2,473 kigo words from an online haiku repository. In addition to generating haiku poems, it can output them vocally together with related sound effects and images retrieved from the WWW. Our experiments demonstrate that the proposed system generates high-quality haikus and that using content-related input and multimedia-rich output is effective for increasing users' satisfaction. We have performed impression evaluation experiments and confirmed that our method is especially useful for generating haikus with higher depth and sound-sharpness, which are two very important categories in professional evaluation of Japanese poetry. Next, haikus generated using the proposed method were evaluated by blog authors and blog readers and again, the proposed method outperformed the baseline. We also measured how the presence or absence of multimedia output influenced the evaluation. While using both vocal output and an image achieved higher scores than text alone, there were cases in which some combinations of effects were evaluated higher than all the effects used together. With our original approach to generating poetry, we wish to show the importance of new media and possibilities that are arising from the utilization of the "wisdom of (web-)crowds" in order to achieve higher standards for AI-generated art.

We address the problem of generating all possible palindromes from a corpus of Ngrams. Palindromes are texts that read the same both ways. Short palindromes ("race car") usually carry precise, significant meanings. Long palindromes are often less meaningful, but even harder to generate. The palindrome generation problem has never been addressed, to our knowledge, from a strictly combinatorial point of view. The main difficulty is that generating palindromes require the simultaneous consideration of two inter-related levels in a sequence: the "character" and the "word" levels. Although the problem seems very combinatorial, we propose an elegant yet non-trivial graph structure that can be used to generate all possible palindromes from a given corpus of Ngrams, with a linear complexity. We illustrate our approach with short and long palindromes obtained from the Google Ngram corpus. We show how we can control the semantics, to some extent, by using arbitrary text corpora to bias the probabilities of certain sets of words. More generally this work addresses the issue of modelling human virtuosity from a combinatorial viewpoint, as a means to understand human creativity.

Many natural phenomena exhibit power law spectra. In particular, so-called 1/f α noise series with α close to 1 (also called pink noise) occur in sound, music and countless human artifacts or natural events, from the fluctuations of the flood levels of the Nile to movements of the stock market. As a consequence, many generative models for 1/f noise have been designed to produce series that look or sound “natural” or “human”. In this paper, we formulate the generation of 1/f series as a hard constraint satisfaction problem, so that 1/f noise generation can be used as an add-on to arbitrary sequence generation problems. We take inspiration from a simple yet beautiful stochastic algorithm invented by Voss and introduce the Voss constraint. We show that Voss’ algorithm can be modeled as a tree of ternary sum constraints, leading to efficient filtering. We illustrate our constraint with a melody generation problem, and show that the addition of the Voss constraint tends indeed to produce sequences whose spectrum have a 1/f distribution, regardless of the other constraints of the problem. We discuss the advantages and limitations of this approach and possible extensions.

The Scaffolded Sound Beehive is an immersive multi-media installation which provides viewers an artistic visual and audio experience of activities in a beehive. Data were recorded in urban beehives and processed using sophisticated pattern recognition, AI technologies, and sonification and computer graphics software. The installation includes an experiment in using Deep Learning to interpret the activities in the hive based on sound and microclimate recording.

This work explores the creation of ambiguous images, i.e., images that may induce multistable perception, by evolutionary means. Ambiguous images are created using a general purpose approach, composed of an expression-based evolutionary engine and a set of object detectors, which are trained in advance using Machine Learning techniques. Images are evolved using Genetic Programming and object detectors are used to classify them. The information gathered during classification is used to assign fitness. In a first stage, the system is used to evolve images that resemble a single object. In a second stage, the discovery of ambiguous images is promoted by combining pairs of object detectors. The analysis of the results highlights the ability of the system to evolve ambiguous images and the differences between computational and human ambiguous images.

Information Aesthetics is an emerging sub-field of Data Visualization that aims to engage the viewers and lure them into decode the visualization. The introduction of self-organising systems can aid the creation of these visual representations through the exploration of emergent patterns. In this paper, we apply a swarm based system as a method to create emergent visualizations of data that convey meaningful information in an inciting way, exploring the boundaries between Data Visualization and Information Aesthetics. The approach is used to visually convey the consumption patterns in 729 Portuguese hypermarkets over the course of two years. The analysis of the experimental results focuses on the ability of the emergent visualizations to communicate information while engaging the viewer with organic visuals.