We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,025 hours of daily-life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 855 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception. Project page: https://ego4d-data.org/

Supersized pre-trained language models have pushed the accuracy of various NLP tasks to a new state-of-the-art (SOTA). Rather than pursuing the reachless SOTA accuracy, most works are pursuing improvement on other dimensions such as efficiency, leading to "Pareto SOTA". Different from accuracy, the metric for efficiency varies across different studies, making them hard to be fairly compared. To that end, this work presents ELUE (Efficient Language Understanding Evaluation), a standard evaluation, and a public leaderboard for efficient NLP models. ELUE is dedicated to depicting the Pareto Front for various language understanding tasks, such that it can tell whether and how much a method achieves Pareto improvement. Along with the benchmark, we also pre-train and release a strong baseline, ElasticBERT, whose elasticity is both static and dynamic. ElasticBERT is static in that it allows reducing model layers on demand. ElasticBERT is dynamic in that it selectively executes parts of model layers conditioned on the input. We demonstrate the ElasticBERT, despite its simplicity, outperforms or performs on par with SOTA compressed and early exiting models. The ELUE benchmark is publicly available at http://eluebenchmark.fastnlp.top/.

In this work, a novel idea is presented for combinatorial optimization problems, a hybrid network, which results in a superior outcome. We applied this method to graph pointer networks [1], expanding its capabilities to a higher level. We proposed a hybrid pointer network (HPN) to solve the travelling salesman problem trained by reinforcement learning. Furthermore, HPN builds upon graph pointer networks which is an extension of pointer networks with an additional graph embedding layer. HPN outperforms the graph pointer network in solution quality due to the hybrid encoder, which provides our model with a verity encoding type, allowing our model to converge to a better policy. Our network significantly outperforms the original graph pointer network for small and large-scale problems increasing its performance for TSP50 from 5.959 to 5.706 without utilizing 2opt, Pointer networks, Attention model, and a wide range of models, producing results comparable to highly tuned and specialized algorithms. We make our data, models, and code publicly available [2].

Abiodun Ogunyemi has been an Uber Eats delivery driver since February 2020. But since March he has been unable to work due to what a union supporting drivers claims is a racially-biased algorithm. Ogunyemi, who is Black, had submitted a photograph of himself to confirm his identity on the app, but when the software failed to recognize him, he was blocked from accessing his account for "improper use of the Uber application." Ogunyemi is one of dozens of Uber drivers who have been prevented from working due to what they say is "racist" facial verification technology. Uber uses Microsoft Face API software on its app to verify drivers' identification, asking drivers to submit new photos on a regular basis.

An international group of researchers has successfully applied AI technology to real-world retinal imagery to detect possible diseases more accurately and on a larger scale. Retinal examinations can detect a number of diseases that affect the eye. Fundus photography is a process of taking photographs of the interior of the eye through the pupil and is a way to screen and monitor such retinal diseases. The introduction of artificial intelligence (AI) technology to fundus photography has improved the platform and enabled it to detect and monitor retinal diseases on a large scale. The Comprehensive AI Retinal Expert (CARE) system was developed by an international group of researchers from Sun Yat-sen University, Beijing Eaglevision Technology (Airdoc), Monash University, University of Miami Miller School of Medicine, Beijing Tongren Eye Centre and Capital Medical University.

It's Ada Lovelace Day and once again we're delighted to introduce you to "50 women in robotics you need to know about"! From the Afghanistan Girls Robotics Team to K.G.Engelhardt who in 1989 founded, and was the first Director of, the Center for Human Service Robotics at Carnegie Mellon, these women showcase a wide range of roles in robotics. We hope these short bios will provide a world of inspiration, in our ninth Women in Robotics list! They are researchers, industry leaders, and artists. Some women are at the start of their careers, while others have literally written the book, the program or the standards.

The effects of climate change could already be impacting 85% of the world's population, an analysis of tens of thousands of scientific studies said Monday. A team of researchers used machine learning to comb through vast troves of research published between 1951 and 2018, and found some 100,000 papers that potentially documented evidence of climate change's effects on the Earth's systems. "We have overwhelming evidence that climate change is affecting all continents -- all systems," study author Max Callaghan said in an interview. He added that there was a "huge amount of evidence" showing the ways in which these impacts are being felt. The researchers taught a computer to identify climate-relevant studies, generating a list of papers on topics from disrupted butterfly migration to heat-related human deaths and forestry cover changes.

Speech summarization is typically performed by using a cascade of speech recognition and text summarization models. End-to-end modeling of speech summarization models is challenging due to memory and compute constraints arising from long input audio sequences. Recent work in document summarization has inspired methods to reduce the complexity of self-attentions, which enables transformer models to handle long sequences. In this work, we introduce a single model optimized end-to-end for speech summarization. We apply the restricted self-attention technique from text-based models to speech models to address the memory and compute constraints. We demonstrate that the proposed model learns to directly summarize speech for the How-2 corpus of instructional videos. The proposed end-to-end model outperforms the previously proposed cascaded model by 3 points absolute on ROUGE. Further, we consider the spoken language understanding task of predicting concepts from speech inputs and show that the proposed end-to-end model outperforms the cascade model by 4 points absolute F-1.

In my previous Nextgov column, I got to interview the general manager of a company that is using artificial intelligence to help the military plan out its maintenance schedules. That program uses AI in a really good way that plays to its strengths, namely its ability to consider thousands of data points, much more than a human ever could, to come up with an action plan for maintenance that maximizes both efficiency and safety. However, when it comes time to actually perform the maintenance, those tasks must be delegated back to a human. But what happens if those physical tasks are also extremely complicated? The CV-22 Osprey is a perfect example of a military aircraft that is both revolutionary and complicated to maintain.

Artificial intelligence has made a disheartening discovery – 85 percent of the world's population has already been affected by human-induced climate change. The findings were made by German scientists, led by Max Callaghan from the Mercator Research Institute on Global Commons and Climate Change, who, according to the study, trained the system to'identify, evaluate and summarize scientific publications on climate change and its consequences.' Researchers used machine learning to sift through data published from 1951 through 2018 and found more than 100,000 studies with evidence that shows 80 percent of Earth's inhabited land has been impacted by climate change. The results also uncovered an'attribution gap' around the globe, where evidence is is distributed unequally across countries - 'evidence for potentially attributable impacts are twice as prevalent in high-income than in low-income countries,' according to the study. Artificial intelligence has made a disheartening discovery – 85 percent of the world's population has already been affected by human-induced climate change.