Due to the recent achievements of artificial neural networks across many different tasks (such as face recognition, object detection and Go), deep learning has become extremely popular. This post aims to be a starting point for those interested in learning more about it. If you already have a basic understanding of linear algebra, calculus, probability and programming: I recommend starting with Stanford's CS231n. The course notes are comprehensive and well-written. The slides for each lesson are also available, and even though the accompanying videos were removed from the official site, re-uploads are quite easy to find online.

Today, the National Science Foundation (NSF) announced that it will invest more than 400 million over the next seven years to support fundamental wireless research and to develop platforms for advanced wireless research in support of the White House's Advanced Wireless Research Initiative. These investments will support the research community in experimenting with and testing novel technologies, applications and services capable of making wireless communication faster, smarter, more responsive and more robust. In the last decade, wireless usage across the U.S. has expanded dramatically, with nearly 350 million smartphones, connected tablets and wearable devices in use -- more than double the number from a decade ago -- carrying more than 100,000 times the traffic they supported in 2008. Experts anticipate as many as 200 billion connected devices globally by 2020. The need for ultra-high-speed, high-bandwidth and low-latency (rapid-response) wireless connectivity will only increase.

Tesla's controversial Autopilot feature was not to blame for a crash at a turnpike in Pennsylvania earlier this month, Elon Musk has claimed. Logs released by the electric car manufacturer show that the feature was not turned on at the time of the crash, despite reports that it may have played a role. The accident came weeks after a Tesla Model S collided with a tractor-trailer on a freeway in Williston, Florida, after it failed to detect the white trailer against the sky. Tesla's auto-pilot system did not cause a crash that'destroyed' a Model X car (pictured) after it suddenly swerved off a road at 60mph and hit some wooden guard rail posts. The 40-year-old driver Joshua Brown died in the accident.

Object queries are essential in information seeking and decision making in vast areas of applications. However, a query may involve complex conditions on objects and sets, which can be arbitrarily nested and aliased. The objects and sets involved as well as the demand---the given parameter values of interest---can change arbitrarily. How to implement object queries efficiently under all possible updates, and furthermore to provide complexity guarantees? This paper describes an automatic method. The method allows powerful queries to be written completely declaratively. It transforms demand as well as all objects and sets into relations. Most importantly, it defines invariants for not only the query results, but also all auxiliary values about the objects and sets involved, including those for propagating demand, and incrementally maintains all of them. Implementation and experiments with problems from a variety of application areas, including distributed algorithms and probabilistic queries, confirm the analyzed complexities, trade-offs, and significant improvements over prior work.

Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate.

Spectral embedding uses eigenfunctions of the discrete Laplacian on a weighted graph to obtain coordinates for an embedding of an abstract data set into Euclidean space. We propose a new pre-processing step of first using the eigenfunctions to simulate a low-frequency wave moving over the data and using both position as well as change in time of the wave to obtain a refined metric to which classical methods of dimensionality reduction can then applied. This is motivated by the behavior of waves, symmetries of the wave equation and the hunting technique of bats. It is shown to be effective in practice and also works for other partial differential equations -- the method yields improved results even for the classical heat equation.

Functional Magnetic Resonance Imaging (fMRI) is a noninvasive technique for studying brain activity, which receives an increasing attention in the last decade or so. During an fMRI experiment, a series of brain images is acquired, while the subject possibly performs a set of tasks responding to external stimuli. Changes in the measured blood-oxygen-level dependent (BOLD) signal are used to examine different types of activation in the brain. There are several objectives in the analysis of fMRI data, the most common of which are the localization of regions of the brain, that are activated by a task, and the determination of the functional brain connectivity [1, 2]. The localization of the activated areas in the human brain is a challenging "cocktail party" problem, where several people are talking (areas activated) simultaneously behind a wall (skull). Our goal is to distinguish those areas (spatial maps) as well as activation patterns (time courses) through some blind source separation (decomposition) method [3, 4]. Each source is the outcome of a combination of a time course with a spatial map. In fMRI studies of the brain function, the structure of the data involves multiple modes, such as trial, task condition, subject, in addition to the intrinsic dimensions of time and space [5].

We formalize Simplified Boardgames language, which describes a subclass of arbitrary board games. The language structure is based on the regular expressions, which makes the rules easily machine-processable while keeping the rules concise and fairly human-readable.

Sum-Product Networks (SPN) have recently emerged as a new class of tractable probabilistic graphical models. Unlike Bayesian networks and Markov networks where inference may be exponential in the size of the network, inference in SPNs is in time linear in the size of the network. Since SPNs represent distributions over a fixed set of variables only, we propose dynamic sum product networks (DSPNs) as a generalization of SPNs for sequence data of varying length. A DSPN consists of a template network that is repeated as many times as needed to model data sequences of any length. We present a local search technique to learn the structure of the template network. In contrast to dynamic Bayesian networks for which inference is generally exponential in the number of variables per time slice, DSPNs inherit the linear inference complexity of SPNs. We demonstrate the advantages of DSPNs over DBNs and other models on several datasets of sequence data.

The art and science of training neural networks from large data sets in order to make predictions or classifications has experienced a major transition over the past several years. Through popular and growing interest from scientists and engineers, this field of data analysis has come to be called deep learning. Put succinctly, deep learning is the ability of machine learning algorithms to acquire feature hierarchies from data and then persist those features within multiple non-linear layers which comprise the machine's learning center, or neural network. Two years ago, questions were mainly about what deep learning is, and how it might be applied to problems in science, engineering, and finance. Over the past year, however, the climate of interest has changed from a curiosity about what deep learning is, and into a focus on acquiring hardware and software in order to apply deep learning frameworks to specific problems across a wide range of disciplines.