Inspired by human learning, researchers have proposed ordering examples during training based on their difficulty. Both curriculum learning, exposing a network to easier examples early in training, and anti-curriculum learning, showing the most difficult examples first, have been suggested as improvements to the standard i.i.d. training. In this work, we set out to investigate the relative benefits of ordered learning. We first investigate the \emph{implicit curricula} resulting from architectural and optimization bias and find that samples are learned in a highly consistent order. Next, to quantify the benefit of \emph{explicit curricula}, we conduct extensive experiments over thousands of orderings spanning three kinds of learning: curriculum, anti-curriculum, and random-curriculum -- in which the size of the training dataset is dynamically increased over time, but the examples are randomly ordered. We find that for standard benchmark datasets, curricula have only marginal benefits, and that randomly ordered samples perform as well or better than curricula and anti-curricula, suggesting that any benefit is entirely due to the dynamic training set size. Inspired by common use cases of curriculum learning in practice, we investigate the role of limited training time budget and noisy data in the success of curriculum learning. Our experiments demonstrate that curriculum, but not anti-curriculum can indeed improve the performance either with limited training time budget or in existence of noisy data.

The distribution of a neural network's latent representations has been successfully used to detect Out-of-Distribution (OOD) data. Since OOD detection denotes a popular benchmark for epistemic uncertainty estimates, this raises the question of a deeper correlation. This work investigates whether the distribution of latent representations indeed contains information about the uncertainty associated with the predictions of a neural network. Prior work identifies epistemic uncertainty with the surprise, thus the negative log-likelihood, of observing a particular latent representation, which we verify empirically. Moreover, we demonstrate that the output-conditional distribution of hidden representations allows quantifying aleatoric uncertainty via the entropy of the predictive distribution. We analyze epistemic and aleatoric uncertainty inferred from the representations of different layers and conclude with the exciting finding that the hidden repesentations of a deterministic neural network indeed contain information about its uncertainty. We verify our findings on both classification and regression models.

DeepMind, an AI research lab that was bought by Google and is now an independent part of Google's parent company Alphabet, announced a major breakthrough this week that one evolutionary biologist called "a game changer." "This will change medicine," the biologist, Andrei Lupas, told Nature. The breakthrough: DeepMind says its AI system, AlphaFold, has solved the "protein folding problem" -- a grand challenge of biology that has vexed scientists for 50 years. Proteins are the basic machines that get work done in your cells. They start out as strings of amino acids (imagine the beads on a necklace) but they soon fold up into a unique three-dimensional shape (imagine scrunching up the beaded necklace in your hand).

Researchers from all over the world contribute to this repository as a prelude to the peer review process for publication in traditional journals. The articles listed below represent a small fraction of all articles appearing on the preprint server. They are listed in no particular order with a link to each paper along with a brief overview. Links to GitHub repos are provided when available. Especially relevant articles are marked with a "thumbs up" icon.

My Name Is GPT-3 and I Approved This Article. The latest natural-language system generates tweets, pens poetry, summarizes emails, answers trivia questions, translates languages and….

Verdict lists the top five terms tweeted on big data in November 2020 based on data from GlobalData's Influencer Platform. The top tweeted terms are the trending industry discussions happening on Twitter by key individuals (influencers) as tracked by the platform. The massive adoption of artificial intelligence (AI) for driving innovations, top applications of AI, and risks associated with AI were popularly discussed in November. According to an article shared by Dr Omkar Rai, director general of Software Technology Parks of India (STPI), the massive adoption of AI is driving innovations in areas such as health research, data analytics, and robotic assistants, to name a few. Research from UnivDatos Market Insights, a market research firm, finds that AI's contribution to the healthcare sector is expected to grow at a compounded annual growth rate (CAGR) of 41% between 2018 and 2025 and will be worth $26.6bn by 2025.

Apart from the unfortunate pandemic that struck the year 2020, it saw a lot of newer developments in the tech domain. While we covered most of these developments in the form of videos, we also introduced newer segments such as the Simulated Reality, a podcast feature by AIM and videos with renowned data scientists covering their journey in detail. This article lists 15 most popular and most viewed videos that you could binge on. Find our YouTube Channel here. One of the most viewed videos of the year; it covers the most popular technology of the year -- GPT-3.

Computer vision models known as convolutional neural networks can be trained to recognize objects nearly as accurately as humans do. However, these models have one significant flaw: Very small changes to an image, which would be nearly imperceptible to a human viewer, can trick them into making egregious errors such as classifying a cat as a tree. A team of neuroscientists from MIT, Harvard University, and IBM have developed a way to alleviate this vulnerability, by adding to these models a new layer that is designed to mimic the earliest stage of the brain's visual processing system. In a new study, they showed that this layer greatly improved the models' robustness against this type of mistake. "Just by making the models more similar to the brain's primary visual cortex, in this single stage of processing, we see quite significant improvements in robustness across many different types of perturbations and corruptions," says Tiago Marques, an MIT postdoc and one of the lead authors of the study.

Self-driving cars have begun to become a reality in the fields of agriculture, transportation, and military, and the day when ordinary consumers use self-driving cars in their daily lives is quickly approaching. An autonomous vehicle performs necessary operations based on sensor information and AI algorithms. It needs to collect data, plan trajectories, and execute driving routes. These tasks, especially planning and executing trajectories, require non-traditional programming methods, which rely on machine learning techniques in AI. Traditional heuristic algorithms in computer science can be used for path planning and control, such as Bellman-Ford algorithm and Dijkstra algorithm.

Within any living organism, there are thousands of different proteins, each with its own unique shape. For decades, the exact formation of those shapes has been a pain for scientists to figure out. How exactly does a protein, which starts as a string of amino acids, fold itself into the funky 3D shapes you might recognize from diagrams? AlphaFold, an AI from DeepMind, may have an answer. It can predict, with heretofore unseen accuracy, the shape a protein will take.