Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.

Appearance changes based on lighting, seasonal, and weather conditions provide a significant challenge for outdoor robots relying on machine learning models for perception. While providing high performance in their training domain, visual shifts occurring in the environment can result in significant deviations from the training distribution, severely reducing accuracy during deployment. Commonly, this challenge is partially counteracted by employing additional training methods to render these models invariant to their application domain [1]. For scenarios where labelled data is unavailable in the target domain, the problem can be addressed in the context of unsupervised domain adaptation [2], [3]. Recent stateof-the-art approaches which address this challenge operate by training deep neural networks within an adversarial domain adaptation (ADA) framework. These approaches are characterised by the optimisation of potentially multiple encoders with the objective to confuse a domain discriminator operating on their output [3], [4], [5] in additional to their main objective. The main intuition behind this framework is that by training the encoder to obtain a domain invariant embedding, we allow the main supervised task to be robust to changes in the application domain.

Sparse dictionary learning (SDL) has become a popular method for adaptively identifying parsimonious representations of a dataset, a fundamental problem in machine learning and signal processing. While most work on SDL assumes a training dataset of independent and identically distributed samples, a variant known as convolutional sparse dictionary learning (CSDL) relaxes this assumption, allowing more general sequential data sources, such as time series or other dependent data. Although recent work has explored the statistical properties of classical SDL, the statistical properties of CSDL remain unstudied. This paper begins to study this by identifying the minimax convergence rate of CSDL in terms of reconstruction risk, by both upper bounding the risk of an established CSDL estimator and proving a matching information-theoretic lower bound. Our results indicate that consistency in reconstruction risk is possible precisely in the `ultra-sparse' setting, in which the sparsity (i.e., the number of feature occurrences) is in $o(N)$ in terms of the length N of the training sequence. Notably, our results make very weak assumptions, allowing arbitrary dictionaries and dependent measurement noise. Finally, we verify our theoretical results with numerical experiments on synthetic data.

We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorization by design" of LSTM matrix into the product of two smaller matrices, and the second one is partitioning of LSTM matrix, its inputs and states into the independent groups. Both approaches allow us to train large LSTM networks significantly faster to the near state-of the art perplexity while using significantly less RNN parameters.

Expressive efficiency refers to the relation between two architectures A and B, whereby any function realized by B could be replicated by A, but there exists functions realized by A, which cannot be replicated by B unless its size grows significantly larger. For example, it is known that deep networks are exponentially efficient with respect to shallow networks, in the sense that a shallow network must grow exponentially large in order to approximate the functions represented by a deep network of polynomial size. In this work, we extend the study of expressive efficiency to the attribute of network connectivity and in particular to the effect of "overlaps" in the convolutional process, i.e., when the stride of the convolution is smaller than its filter size (receptive field). To theoretically analyze this aspect of network's design, we focus on a well-established surrogate for ConvNets called Convolutional Arithmetic Circuits (ConvACs), and then demonstrate empirically that our results hold for standard ConvNets as well. Specifically, our analysis shows that having overlapping local receptive fields, and more broadly denser connectivity, results in an exponential increase in the expressive capacity of neural networks. Moreover, while denser connectivity can increase the expressive capacity, we show that the most common types of modern architectures already exhibit exponential increase in expressivity, without relying on fully-connected layers.

Deep neural networks excel in regimes with large amounts of data, but tend to struggle when data is scarce or when they need to adapt quickly to changes in the task. In response, recent work in meta-learning proposes training a meta-learner on a distribution of similar tasks, in the hopes of generalization to novel but related tasks by learning a high-level strategy that captures the essence of the problem it is asked to solve. However, many recent meta-learning approaches are extensively hand-designed, either using architectures specialized to a particular application, or hard-coding algorithmic components that constrain how the meta-learner solves the task. We propose a class of simple and generic meta-learner architectures that use a novel combination of temporal convolutions and soft attention; the former to aggregate information from past experience and the latter to pinpoint specific pieces of information. In the most extensive set of meta-learning experiments to date, we evaluate the resulting Simple Neural AttentIve Learner (or SNAIL) on several heavily-benchmarked tasks.

Time-series data arise in many fields including finance, signal processing, speech recognition and medicine. A standard approach to time-series problems usually requires manual engineering of features which can then be fed into a machine learning algorithm. Engineering of features generally requires some domain knowledge of the discipline where the data has originated from. For example, if one is dealing with signals (i.e. A similar situation arises in image classification, where manually engineered features (obtained by applying a number of filters) could be used in classification algorithms.

The Department of Veterans Affairs has announced a research partnership with Alphabet subsidiary DeepMind that will tackle issues concerning patient deterioration during hospital care. Using a dataset comprised of 700,000 historical, de-personalized health records, the machine learning platform will help the VA identify risk factors for deterioration while predicting its onset. "Medicine is more than treating patients' problems," VA Secretary David J. Shulkin, MD, said in a statement. "Clinicians need to be able to identify risks to help prevent disease. This collaboration is an opportunity to advance the quality of care for our nation's veterans by predicting deterioration and applying interventions early."

Artificial intelligence is a very broad term that dates back to 1956 and describes efforts to get computers to do things that could previously only be done by humans. It occurs frequently in science fiction, yet today's excitement is focused on one particular subfield of AI known as "machine learning," which involves teaching machines to do things by example – as opposed to, say, "expert systems," which rely on rules and knowledge distilled from human experts. The excitement, in fact, is mostly centered on a specific machine learning technique known as "deep learning," in which software simulations of simple models of the human brain are trained to do things by showing them large numbers of examples. "Neural networks," as these simulations are called, have been around for a while, but "deep" networks, which are more sophisticated and can be trained to recognize more subtle differences, have become far more capable in recent years. In other words, deep learning is just one very specific example of a subfield of AI, but the excitement around this particular subfield stems from its ability to handle a very wide range of problems, from image recognition to language translation to transcribing speech.

Google subsidiary DeepMind announced today that it's working with the U.S. Department of Veterans Affairs to use machine learning in an attempt to predict when patients will deteriorate during a hospital stay. Deterioration (when a patient's condition worsens) is a significant issue, since care providers can miss warning signs for potentially lethal conditions that arise as part of other treatment. DeepMind and the VA aim to tackle Acute Kidney Injury (AKI), which, as the name implies, occurs when a person's kidneys temporarily stop working as well as they should. That can mean kidney failure, or just injury that reduces kidney function. AKI can be fatal if untreated.