Computational Creativity: The Role of the Transformer

Over the years, computers have become increasingly sophisticated in their ability to identify more and more complex patterns. The field of computational creativity, a multidisciplinary endeavour to build software that can assist humans in a variety of tasks in the arts, science and the humanities, has seen much progress since the early days of computers where instructions had to be explicitly programmed. In this article, we will attempt to unravel some of the recent developments in generative modelling that have shown significant improvements in computers' ability to generate useful patterns that appeal to human observers. In particular, one type of neural network architecture, the Transformer, will be discussed in detail with regard to its ability to capture longer-term dependencies in text, music and images. Some future directions that this technology could lead to are also discussed.