We present a system for real-time lyrical improvisation between a human and a robot in the style of hip hop. Our system takes vocal input from a human rapper, analyzes the semantic meaning, and generates a response that is rapped back by a robot over a musical groove. Previous work with real-time interactive music systems has largely focused on instrumental output, and vocal interactions with robots have been explored, but not in a musical context. Our generative system includes custom methods for censorship, voice, rhythm, rhyming and a novel deep learning pipeline based on phoneme embeddings. The rap performances are accompanied by synchronized robotic gestures and mouth movements. Key technical challenges that were overcome in the system are developing rhymes, performing with low-latency and dataset censorship. We evaluated several aspects of the system through a survey of videos and sample text output. Analysis of comments showed that the overall perception of the system was positive. The model trained on our hip hop dataset was rated significantly higher than our metal dataset in coherence, rhyme quality, and enjoyment. Participants preferred outputs generated by a given input phrase over outputs generated from unknown keywords, indicating that the system successfully relates its output to its input.

Shimon the singing songwriting robot has been taught to write his own lyrics by studying tens of thousands of songs written by the musical greats. Developed by researchers from the Georgia Tech Center for Music Technology, the robot collaborates with human musicians and even has an album out in the spring. The robot was given a dataset of 50,000 lyrics covering all genres including rock, hip-hop, jazz and progressive as part of its song writing education. As well as writing the lyrics the robot can sing them and dance while performing with'his band' made up of Georgia Tech students and researchers. Professor Gil Weinberg, creator of Schimon said he works with humans to create music, they are a mixture of songs made by human and robot together.

In this episode, our interviewer Audrow Nash speaks to Gil Weinberg, Professor in Georgia Tech's School of Music and the founding director of the Georgia Tech Center for Music Technology. Weinberg leads a research lab called the Robotic Musicianship group, which focuses on developing artificial creativity and musical expression for robots and on augmented humans. Weinberg discusses several of his improvisational robots and how they work, including Shimon, a multi-armed robot marimba player, as well as his work in prosthetic devices for musicians. Below is a video that includes Shimon on marimba, Jason Barnes playing drums with a prostetic, and Prof. Gil Weinberg on bass guitar. Gil Weinberg is a professor in Georgia Tech's School of Music and the founding director of the Georgia Tech Center for Music Technology, where he leads the Robotic Musicianship group.

This Friday marks the start of the fifth annual Atlanta Science Festival, kicking off with Rise Up, Robots, a variety show featuring an assortment of robotic performers. One of those performers will be Shimon, a marimba-playing robot that uses machine learning to develop new and inventive compositions. Shimon was created by Gil Weinberg, professor and founding director of Georgia Tech's Center for Music Technology. Many at Tech have heard of Shimon and its ability to improvise jazz melodies. This Friday, though, the musical robot will tread uncharted territory, showcasing a new rock composition composed by Zach Kondak, a graduate student in music technology, who will also play drums and guitar.

"Have you heard the rumor about butter? Go ahead and roll your eyes. This groaner is just one of the many, many terrible jokes that Amazon's "personal assistant" software, Alexa, will tell you -- if you ask. But Alexa can do a lot more than make bad puns. Many people start their mornings by asking Alexa for the weather forecast or the latest news. A device that houses the software can also play music from your favorite playlists, keep a shopping list, order takeout food, answer trivia questions, send voice messages and even run "smart" home controls like thermostats. Alexa is a form of artificial intelligence, or AI for short.

The Georgia Tech Center for Music Technology, led by Gil Weinberg, has a reputation for doing incredible musical things with robots, with a mix of creativity and technical expertise in robotics and AI. We've seen projects like a cybernetic second arm for a drummer, a cybernetic third arm (!) for a drummer, and a bunch of interesting research on ways that robots can dynamically collaborate with humans in the context of improvisational music. That last thing usually features Shimon, a four-armed expressive robotic marimba player, which can analyze music in real time and improvise along with human performers. It's an impressive thing to watch, but Shimon's talents were mostly restricted to riffing on what other human musicians were doing. Now, Shimon has leveraged deep learning to create structured and coherent and totally unique compositions of its very own.

A robot developed by researchers at Georgia Tech has a knack for music that would have made Beethoven jealous. Shimon is robot that composes and plays his own beats. He uses a process called deep neural learning along with artificial intelligence to decode different genres of music eventually creating his own chimes on the marimba. Dr. Gil Weinberg, the director of the Georgia Tech Center for Music Technology, created Shimon nearly 10 years ago, but now the robot musician has learned how to do things on his own. Weinberg's goal is to use the technology to collaborate with humans rather than replace them.

In a first, scientists have developed a marimba-playing robot that uses artificial intelligence to create its own music inspired by the works of musicians like Beethoven and Mozart. The robot with four arms and eight sticks writes and plays its own compositions on a marimba, using a database of well-known pop, classical and jazz artists. Researchers at Georgia Institute of Technology in the US fed the robot nearly 5,000 complete songs – from Beethoven to the Beatles to Lady Gaga to Miles Davis. They worked with the robot named'Shimon' for seven years, enabling it to listen to music played by humans and improvise over pre-composed chord progressions. Shimon is now a solo composer generating the melody and harmonic structure on its own, researchers said.

Having four arms would be an advantage for any musician, but they are just one of the many unique features of Shimon, the marimba playing robot. The machine has used its artificial intelligence and deep learning algorithms to analyse more than two million motifs, riffs and licks of music to create its own masterpiece. Aside from giving the machine the first four bars to use as a starting point, no humans are involved in either the composition or the performance of the music. Shimon (pictured) has used its artificial intelligence and deep learning algorithms to analyse over two million motifs, riffs and licks of music to create and perform its own masterpiece. Shimon is the creation of Mason Bretan, a PhD student at Georgia Tech, that uses eight sticks to play the wooden percussion instrument. He has worked with Shimon for seven years, enabling it to'listen' to music played by humans and improvise over composed chord progressions.

Researchers fed the robot nearly 5,000 complete songs -- from Beethoven to the Beatles to Lady Gaga to Miles Davis -- and more than 2 million motifs, riffs and licks of music. Aside from giving the machine a seed, or the first four measures to use as a starting point, no humans are involved in either the composition or the performance of the music. The first two compositions are roughly 30 seconds in length. Ph.D. student Mason Bretan is the man behind the machine. He's worked with Shimon for seven years, enabling it to "listen" to music played by humans and improvise over pre-composed chord progressions.