Goto

Collaborating Authors

 new record


VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

Neural Information Processing Systems

We present a framework for learning multimodal representations from unlabeled data using convolution-free Transformer architectures. Specifically, our Video-Audio-Text Transformer (VATT) takes raw signals as inputs and extracts multimodal representations that are rich enough to benefit a variety of downstream tasks. We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and text-to-video retrieval. Furthermore, we study a modality-agnostic single-backbone Transformer by sharing weights among the three modalities. We show that the convolution-free VATT outperforms state-of-the-art ConvNet-based architectures in the downstream tasks. Especially, VATT's vision Transformer achieves the top-1 accuracy of 82.1% on Kinetics-400, 83.6% on Kinetics-600, 72.7% on Kinetics-700, and 41.1% on Moments in Time, new records while avoiding supervised pre-training. Transferring to image classification leads to 78.7% top-1 accuracy on ImageNet compared to 64.7% by training the same Transformer from scratch, showing the generalizability of our model despite the domain gap between videos and images. VATT's audio Transformer also sets a new record on waveform-based audio event recognition by achieving the mAP of 39.4% on AudioSet without any supervised pre-training.


A Minecraft Movie just set a new record with the biggest opening ever for a video game adaptation in the US

Engadget

A Minecraft Movie has reportedly surpassed the record previously set by 2023's The Super Mario Bros. Movie for the biggest ever domestic box office opening of a video game adaptation. The new movie, which was released in theaters on Friday, raked in 157 million in the US in its opening weekend, according to The Hollywood Reporter. A Minecraft Movie is doing well internationally, too; THR reports that it's earned 301M altogether in its global debut. The Super Mario Bros. Movie pulled in 146 million in its domestic opening and 377 million globally. A Minecraft Movie stars Jack Black, Sebastian Hansen, Emma Myers, Jason Momoa, Danielle Brooks and Jennifer Coolidge.


VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

Neural Information Processing Systems

We present a framework for learning multimodal representations from unlabeled data using convolution-free Transformer architectures. Specifically, our Video-Audio-Text Transformer (VATT) takes raw signals as inputs and extracts multimodal representations that are rich enough to benefit a variety of downstream tasks. We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and text-to-video retrieval. Furthermore, we study a modality-agnostic single-backbone Transformer by sharing weights among the three modalities. We show that the convolution-free VATT outperforms state-of-the-art ConvNet-based architectures in the downstream tasks. Especially, VATT's vision Transformer achieves the top-1 accuracy of 82.1% on Kinetics-400, 83.6% on Kinetics-600, 72.7% on Kinetics-700, and 41.1% on Moments in Time, new records while avoiding supervised pre-training.


One million robots work in car industry worldwide – new record

Robohub

The automotive industry has the largest number of robots working in factories around the world: Operational stock hit a new record of about one million units. This represents about one third of the total number installed across all industries. "The automotive industry effectively invented automated manufacturing," says Marina Bill, President of the International Federation of Robotics. "Today, robots are playing a vital role in enabling this industry's transition from combustion engines to electric power. Robotic automation helps car manufacturers manage the wholesale changes to long-established manufacturing methods and technologies."

  Country:
  Industry:

DeepMind's AI beats new record

#artificialintelligence

Google's DeepMind has beaten a 50-year-old record, contributing to a major development in the field of machine learning. Researchers at the lab trained a new version of its board game-playing AI, AlphaZero, to figure out a faster way to do matrix multiplication, a fundamental problem in computing that powers everything from displaying images on a screen to simulating complex physics. Speeding up the calculation could have "a big impact on thousands of everyday computer tasks," according to MIT Technology Review, and cut costs and save energy.


The application of an Artificial Neuron on the Iris Dataset in Python

#artificialintelligence

Artificial Neural Networks (ANNs) are extremely powerful. Recent developments brought scientists to create NNs with more connections than a human brain. To give you an idea, it is estimated that an average brain has 86 billion neurons and 100 billion synapses. On the other hand, the largest NN in 2022, "Megatron-Turing NGL 530B (MT-NGL)", a monolithic transformer language model, has 530 billion parameters. Still, the human brain is proficient in more than one field, MT-NGL is only specialized in language processing.


IFR predicts 'Top 5 Robot Trends of 2022' as total industrial robots sales reach a new record

#artificialintelligence

The International Federation of Robotics has published what it predicts will be the top five industry trends of 2022. The main theme underlying the trends will be that robots with new features and functions will capture new areas and create new markets. The operational stock of industrial robots hit a new record of about 3 million units worldwide – increasing by 13 percent on average each year (2015-2020). Milton Guerry, president of the IFR, says: "Transformation for robotic automation is picking up speed across traditional and new industries. More and more companies are realizing the numerous advantages robotics provides for their businesses."


An AI Was Taught to Play the World's Hardest Video Game and Still Couldn't Set a New Record

#artificialintelligence

What's the hardest video game you've ever played? If it wasn't QWOP then let me tell you right know that you don't know how truly difficult a game can be. The deceptively simple running game is so challenging to master that even an AI trained using machine learning still only mustered a top 10 score instead of shattering the record. If you've never played QWOP before, you owe it to yourself to give it a try and see if you can even get your sprinter off the starting line. Developed by Bennett Foddy back in 2008, QWOP was inspired by an '80s arcade game called Track & Field that requires players to mindlessly mashing buttons to win a race.


'Star Trek: Picard' breaks streaming records on CBS All Access – TechCrunch

#artificialintelligence

CBS' streaming service, CBS All Access, credits a trio of high-profile events -- including the premiere of its new Star Trek series, "Star Trek: Picard," as well as the 62nd annual Grammy Awards, not to mention a busy month of football -- with helping it to achieve a new record for subscriber sign-ups in a given month. The company says January 2020 surpassed the service's previous record in February 2019 for subscriber sign-ups. In addition, last week was the second-best sign-up week ever, closely behind the week of the 2019 Super Bowl. Much of the record-setting had to do with the launch of the highly anticipated show, "Star Trek: Picard," which brings back fan-favorite Patrick Stewart as Jean-Luc Picard, now a retired Starfleet Admiral whose quiet life on his family's vineyard is about to be disrupted. The show, set 18 years after the events of the final "Star Trek: The Next Generation" movie, "Star Trek: Nemesis," not only capitalizes on Stewart's draw, it also brings back previous "Star Trek" actors including Brent Spiner (Data), Jeri Ryan (Seven of Nine), Marina Sirtis (Troi), and Jonathan Frakes (Riker). But unlike other reboots, which hope nostalgia alone will bring the viewers, "Picard's" creators have actually given thought to the story the show is trying to tell, resulting in a 95% critics score on Rotten Tomatoes.


Metrics-Driven Machine Learning Development at Salesforce Einstein

#artificialintelligence

Essentially, it's a web app that guides admins through building machine learning models with a few clicks and without having to write any code. As I mentioned, it allows them to make any different objects. We have this machine learning pipeline, an automated machine learning pipeline, in the back end, that trains all the models once we receive the info on the front end. We need to serve many different use cases and we don't have an intimate look ourselves at the data; just some of the example use cases of use, some common ones, binary classification. A lot of customers have subscription-based models, and they might have records of all the customers who have left in the past year or so.