Video-driven 3D facial animation transfer aims to drive avatars to reproduce the expressions of actors. Existing methods have achieved remarkable results by constraining both geometric and perceptual consistency. However, geometric constraints (like those designed on facial landmarks) are insufficient to capture subtle emotions, while expression features trained on classification tasks lack fine granularity for complex emotions. To address this, we propose \textbf{FreeAvatar}, a robust facial animation transfer method that relies solely on our learned expression representation. Specifically, FreeAvatar consists of two main components: the expression foundation model and the facial animation transfer model. In the first component, we initially construct a facial feature space through a face reconstruction task and then optimize the expression feature space by exploring the similarities among different expressions. Benefiting from training on the amounts of unlabeled facial images and re-collected expression comparison dataset, our model adapts freely and effectively to any in-the-wild input facial images. In the facial animation transfer component, we propose a novel Expression-driven Multi-avatar Animator, which first maps expressive semantics to the facial control parameters of 3D avatars and then imposes perceptual constraints between the input and output images to maintain expression consistency. To make the entire process differentiable, we employ a trained neural renderer to translate rig parameters into corresponding images. Furthermore, unlike previous methods that require separate decoders for each avatar, we propose a dynamic identity injection module that allows for the joint training of multiple avatars within a single network.

Creating realistic or stylized facial and lip sync animation is a tedious task. It requires lot of time and skills to sync the lips with audio and convey the right emotion to the character's face. To allow animators to spend more time on the artistic and creative part of the animation, we present Audio2Rig: a new deep learning based tool leveraging previously animated sequences of a show, to generate facial and lip sync rig animation from an audio file. Based in Maya, it learns from any production rig without any adjustment and generates high quality and stylized animations which mimic the style of the show. Audio2Rig fits in the animator workflow: since it generates keys on the rig controllers, the animation can be easily retaken. The method is based on 3 neural network modules which can learn an arbitrary number of controllers. Hence, different configurations can be created for specific parts of the face (such as the tongue, lips or eyes). With Audio2Rig, animators can also pick different emotions and adjust their intensities to experiment or customize the output, and have high level controls on the keyframes setting. Our method shows excellent results, generating fine animation details while respecting the show style. Finally, as the training relies on the studio data and is done internally, it ensures data privacy and prevents from copyright infringement.

At first glance at this video, you'd be forgiven for mistaking it as a clip from Wall-E. But the robot depicted in the footage isn't science fiction - it's very much real. In a video posted by Walt Disney Imagineering, a newly designed bipedal robot walks, struts, dances, and emotes in an impressive display of engineering prowess. The bot also shows off its human interaction skills as it reacts to those around it and even walks behind two children pulling it on a lead. With its expressive head and wiggly antenna, the unnamed robot has been designed to bring the creative designs of animators into the real world using machine learning.

A shortage of artists in the animation industry could lead studios to seek out AI companies for a solution. But, according to PC Mag, Netflix is already taking steps toward plugging that labor hole. The animated work in question is a Netflix Japan short called Dog & Boy. Attack on Titan's Wit Studio employed Rinna, an AI character and scenario company, to create the backgrounds for the anime in question. Netflix Japan admitted in a blog post that this project was an experiment.

Recently, "AI" machine-learning technologies have been creeping their way into artistic fields in both entertaining and harmful ways. While some AI content creators are just making videos for harmless fun, others, like the creators of a recent AI-generated anime short, wrongfully believe they've democratized the animation industry when they've really just come up with a more technologically demanding method of plagiarizing other artists. Earlier this week, Corridor Digital, a Los Angeles-based production studio that creates pop culture YouTube videos, uploaded a video called "Anime Rock, Paper, Scissors." Written and directed by Niko Pueringer and Sam Gorski, it revolves around two twins vying for the throne left vacant by their recently deceased father. By leveraging the machine-learning text-to-image model Stable Diffusion, Corridor Digital gave camera footage filmed in front of a green screen a dramatic anime-like appearance.

Can a zebra walk on the surface of the moon alongside an astronaut? Can high rise buildings that are 80 stories in the middle of the Saharan desert sway dangerously? None of these have ever occurred in the past. What stops anyone from creating simulations to demonstrate such possibilities; what if they are possibilities in the future? While text-to-image capabilities have been remarkable for "visualizing" ideas and simulations, image-to-videography will become the next trend.

Back in 1982, computers meant one of two things in the popular imagination. Either they were room-sized machines used by the military-industrial complex to crunch data on stuff like nuclear wars and stock markets, or they were fridge-sized arcade games such as Space Invaders and Pac-Man. Kraftwerk were singing about home computers, but if you owned one at all, it was probably a Sinclair ZX81, which was only marginally more sophisticated than a calculator. And yet, that summer, cinemagoers were catapulted into the digital future. Few appreciated it at the time but with 40 years' hindsight, Steven Lisberger's sci-fi adventure Tron was the shape of things to come: in cinema, in real life, and in virtual life.

"Knowing how powerful machine learning has become, it's just a matter of time before it completely takes over the animation industry." In recent years, deep learning has increased the modern scope of animation, making it more accessible and powerful than before. Artificial intelligence has become a shiny new weapon in the creator's arsenal. The advancement of hardware and AI has blurred the lines between virtual and real characters(eg: movies like Alita). Something that could have taken hours to perform by animators is being done by automation in minutes.

Amit Katwala, senior editor at Wired UK, explains how animators are using technology to voice characters on long-running shows like The Simpsons.

In this work, we propose a novel deep learning framework that can generate a vivid dance from a whole piece of music. In contrast to previous works that define the problem as generation of frames of motion state parameters, we formulate the task as a prediction of motion curves between key poses, which is inspired by the animation industry practice. The proposed framework, named DanceNet3D, first generates key poses on beats of the given music and then predicts the in-between motion curves. DanceNet3D adopts the encoder-decoder architecture and the adversarial schemes for training. The decoders in DanceNet3D are constructed on MoTrans, a transformer tailored for motion generation. In MoTrans we introduce the kinematic correlation by the Kinematic Chain Networks, and we also propose the Learned Local Attention module to take the temporal local correlation of human motion into consideration. Furthermore, we propose PhantomDance, the first large-scale dance dataset produced by professional animatiors, with accurate synchronization with music. Extensive experiments demonstrate that the proposed approach can generate fluent, elegant, performative and beat-synchronized 3D dances, which significantly surpasses previous works quantitatively and qualitatively.