A deep learning technique to generate real-time lip sync for live 2-D animation

Live 2-D animation is a fairly new and powerful form of communication that allows human performers to control cartoon characters in real time while interacting and improvising with other actors or members of an audience. Recent examples include Stephen Colbert interviewing cartoon guests on The Late Show, Homer answering live phone-in questions from viewers during a segment of The Simpsons, Archer talking to a live audience at ComicCon, and the stars of Disney's Star vs. The Forces of Evil and My Little Pony hosting live chat sessions with fans via YouTube or Facebook Live. Producing realistic and effective live 2-D animations requires the use of interactive systems that can automatically transform human performances into animations in real time. A key aspect of these systems is attaining a good lip sync, which essentially means that the mouths of animated characters move appropriately when speaking, mimicking the movements observed in the mouths of performers.