Consider a prediction setting where a few inputs (e.g., satellite images) are expensively annotated with the prediction targets (e.g., crop types), and many inputs are cheaply annotated with auxiliary information (e.g., climate information). How should we best leverage this auxiliary information for the prediction task? Empirically across three image and time-series datasets, and theoretically in a multi-task linear regression setting, we show that (i) using auxiliary information as input features improves in-distribution error but can hurt out-of-distribution (OOD) error; while (ii) using auxiliary information as outputs of auxiliary tasks to pre-train a model improves OOD error. To get the best of both worlds, we introduce In-N-Out, which first trains a model with auxiliary inputs and uses it to pseudolabel all the in-distribution inputs, then pre-trains a model on OOD auxiliary outputs and fine-tunes this model with the pseudolabels (self-training). We show both theoretically and empirically that In-N-Out outperforms auxiliary inputs or outputs alone on both in-distribution and OOD error.

Earlier this fall, A.I. ethicist Timnit Gebru submitted a paper for consideration at an academic conference about predictive language models: on their environmental cost, and how they could learn racist and sexist language and also spread misinformation. Since she was working for Google, the company first wanted to review the paper--which Gebru wrote with several of her colleagues--and sign off on it. She was then told by senior managers that the paper didn't meet Google's publication bar, and that she should retract it or remove the names of Google employees. Gebru wanted more clarity on why they wanted it retracted and said that if Google couldn't provide that information, she would resign. This kicked off a few days of wrangling and several intense emails--until a manager emailed Gebru's boss, saying they had accepted her resignation.

The fourth-annual Midas List Europe, produced by Forbes in partnership with TrueBridge Capital Partners, has arrived, and we're excited to share the top companies that drove the portfolios of this year's top European venture capitalists. The outlook for the European venture market may have been cloudy at the beginning of the global pandemic as recessionary cutbacks loomed and the IPO window narrowed, but European startups and investors have since bounced back. A wide variety of tech-based startups have been able to ride the tailwinds of the crisis, with new areas of everyday life benefitting from the transition to a technology-driven environment. Evidentially, investors remain clear-eyed and eager to invest in growth and innovation on either side of the pond with European VC deal value – and potentially fundraising – on pace to set new annual records. Here are the top ten companies that acted as key drivers behind this year's Midas List Europe: It's been a boom year for Stockholm-based Spotify, which is making its third consecutive appearance as the #1 driver on the Midas List Europe and fourth appearance overall.

Successive lockdowns imposed across the globe and travel restrictions accelerated digital transformation at workplaces and for essential services. Unable to step out, people turned to online portals and apps for most tasks including shopping, learning and banking. Healthcare gained importance, driving more people to get insurance and firms also embraced digitisation to serve consumers, globally and in the Middle East. But interacting with consumers and verifying claims online, in order to ensure contactless service, has its challenges when cybercrime is surging as quickly as the tech-savvy economy. Saudi Arabia's cooperative insurer Tawuniya also found itself vulnerable, at a time when 95% firms in the kingdom were reportedly targeted by cybercrooks.

Climate change is one of the greatest threats humans have ever faced, with increasingly severe consequences feared as sea levels rise, ecosystems falter, and natural disasters multiply. Tackling climate change is a huge and complex challenge, where it's hoped that AI-powered efforts can play an equally huge and beneficial role. Organizers of NeurIPS 2020 (Conference on Neural Information Processing Systems) see machine learning (ML) as an invaluable tool in the fight against climate change. A wide array of applications and techniques are already being explored, from smart electric grid design to satellite-tracking of greenhouse gas emissions and countless others. Last Friday, NeurIPS 2020 partnered with Climate Change AI (CCAI) -- an organization of researchers, engineers, entrepreneurs, investors, policymakers, companies and NGOs aiming to catalyze impactful work at the intersection of climate change and machine learning -- to host the Tackling Climate Change with ML Workshop, which explored how the ML community could collaborate with other fields and practitioners in this fight. The all-virtual format of NeurIPS 2020, which ran December 6-12, provided a unique opportunity to foster cross-pollination between ML researchers and experts across diverse fields.

Genetic pathways usually encode molecular mechanisms that can inform targeted interventions. It is often challenging for existing machine learning approaches to jointly model genetic pathways (higher-order features) and variants (atomic features), and present to clinicians interpretable models. In order to build more accurate and better interpretable machine learning models for genetic medicine, we introduce Pathway Augmented Nonnegative Tensor factorization for HighER-order feature learning (PANTHER). PANTHER selects informative genetic pathways that directly encode molecular mechanisms. We apply genetically motivated constrained tensor factorization to group pathways in a way that reflects molecular mechanism interactions. We then train a softmax classifier for disease types using the identified pathway groups. We evaluated PANTHER against multiple state-of-the-art constrained tensor/matrix factorization models, as well as group guided and Bayesian hierarchical models. PANTHER outperforms all state-of-the-art comparison models significantly (p<0.05). Our experiments on large scale Next Generation Sequencing (NGS) and whole-genome genotyping datasets also demonstrated wide applicability of PANTHER. We performed feature analysis in predicting disease types, which suggested insights and benefits of the identified pathway groups.

The Explainable Abstract Trains Dataset is an image dataset containing simplified representations of trains. It aims to provide a platform for the application and research of algorithms for justification and explanation extraction. The dataset is accompanied by an ontology that conceptualizes and classifies the depicted trains based on their visual characteristics, allowing for a precise understanding of how each train was labeled. Each image in the dataset is annotated with multiple attributes describing the trains' features and with bounding boxes for the train elements.

Expressivity plays a fundamental role in evaluating deep neural networks, and it is closely related to understanding the limit of performance improvement. In this paper, we propose a three-pipeline training framework based on critical expressivity, including global model contraction, weight evolution, and link's weight rewiring. Specifically, we propose a pyramidal-like skeleton to overcome the saddle points that affect information transfer. Then we analyze the reason for the modularity (clustering) phenomenon in network topology and use it to rewire potential erroneous weighted links. We conduct numerical experiments on node classification and the results confirm that the proposed training framework leads to a significantly improved performance in terms of fast convergence and robustness to potential erroneous weighted links. The architecture design on GNNs, in turn, verifies the expressivity of GNNs from dynamics and topological space aspects and provides useful guidelines in designing more efficient neural networks.

Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, data science, and signal processing. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to initialize other more sophisticated algorithms to improve performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and methods of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional $\ell_2$ perturbation analysis, we present a systematic $\ell_{\infty}$ and $\ell_{2,\infty}$ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful "leave-one-out" analysis framework.

How can we track synchronized behavior in a stream of time-stamped tuples, such as mobile devices installing and uninstalling applications in the lockstep, to boost their ranks in the app store? We model such tuples as entries in a streaming tensor, which augments attribute sizes in its modes over time. Synchronized behavior tends to form dense blocks (i.e. subtensors) in such a tensor, signaling anomalous behavior, or interesting communities. However, existing dense block detection methods are either based on a static tensor, or lack an efficient algorithm in a streaming setting. Therefore, we propose a fast streaming algorithm, AugSplicing, which can detect the top dense blocks by incrementally splicing the previous detection with the incoming ones in new tuples, avoiding re-runs over all the history data at every tracking time step. AugSplicing is based on a splicing condition that guides the algorithm (Section 4). Compared to the state-of-the-art methods, our method is (1) effective to detect fraudulent behavior in installing data of real-world apps and find a synchronized group of students with interesting features in campus Wi-Fi data; (2) robust with splicing theory for dense block detection; (3) streaming and faster than the existing streaming algorithm, with closely comparable accuracy.