Researchers train AI to mimic 20 acrobatic moves from YouTube videos

Researchers at the University of California, Berkeley have created a framework for teaching artificial intelligence systems to learn motion from being shown video clips on YouTube. The framework incorporates computer vision and reinforcement learning to train AI skills from videos. Altogether the team was able to train AI to perform more than 20 acrobatic tasks like cartwheels, handsprings, backflips, and some martial arts. The method does not require the use of motion capture video, the kind often used to transfer human action to digital forms, such as the movement of LeBron James incorporated into NBA 2K18 or the performance of Andy Serkis as Gollum from Lord of the Rings. The framework works by first ingesting the video to understand the poses seen in each video frame; then a simulated character is trained to imitate the movement using reinforcement learning.