AI Delivers a New Creative Direction for the Musicians

LyricJam, a real-time system that uses artificial intelligence (AI) to generate lyric lines for live instrumental music, was created by members of the University's Natural Language Processing Lab. The lab, led by Olga Vechtomova, a Waterloo Engineering professor cross-appointed in Computer Science, has been researching creative applications of AI for several years. The lab's initial work led to the creation of a system that learns musical expressions of artists and generates lyrics in their style. Recently, Vechtomova, along with Waterloo graduate students Gaurav Sahu and Dhruv Kumar, developed technology that relies on various aspects of music such as chord progressions, tempo, and instrumentation to synthesize lyrics reflecting the mood and emotions expressed by live music. As a musician or a band plays instrumental music, the system continuously receives the raw audio clips, which the neural network processes to generate new lyric lines.