Many real-world time-series analysis problems are characterised by scarce data. Solutions typically rely on hand-crafted features extracted from the time or frequency domain allied with classification or regression engines which condition on this (often low-dimensional) feature vector. The huge advances enjoyed by many application domains in recent years have been fuelled by the use of deep learning architectures trained on large data sets. This paper presents an application of deep learning for acoustic event detection in a challenging, data-scarce, real-world problem. Our candidate challenge is to accurately detect the presence of a mosquito from its acoustic signature. We develop convolutional neural networks (CNNs) operating on wavelet transformations of audio recordings. Furthermore, we interrogate the network's predictive power by visualising statistics of network-excitatory samples. These visualisations offer a deep insight into the relative informativeness of components in the detection problem. We include comparisons with conventional classifiers, conditioned on both hand-tuned and generic features, to stress the strength of automatic deep feature learning. Detection is achieved with performance metrics significantly surpassing those of existing algorithmic methods, as well as marginally exceeding those attained by individual human experts.

Deep learning has lately been shown to be a very powerful tool for a wide range of problems, from image segmentation to machine translation. Despite its success, many of the techniques developed by practitioners of artificial neural networks (ANNs) are heuristics without theoretical guarantees. Perhaps most notably, the power of feedforward networks with many layers (deep networks) has not been fully explained. The goal of this paper is to shed more light on this question and to suggest heuristics for how deep is deep enough. It is well-known [1-3] that nonlinear neural networks with a single hidden layer can approximate any function under reasonable assumptions, but it is possible that the networks required will be extremely large. Recent authors have shown that some functions can be approximated by deeper networks much more efficiently (i.e. with fewer neurons) than by shallower ones.

Probabilistic modeling enables combining domain knowledge with learning from data, thereby supporting learning from fewer training instances than purely data-driven methods. However, learning probabilistic models is difficult and has not achieved the level of performance of methods such as deep neural networks on many tasks. In this paper, we attempt to address this issue by presenting a method for learning the parameters of a probabilistic program using backpropagation. Our approach opens the possibility to building deep probabilistic programming models that are trained in a similar way to neural networks.

My first recollection of an effective Deep Learning system that used feedback loops where in "Ladder Networks". In an architecture developed by Stanford called "Feedback Networks", the researchers explored a different kind of network that feeds back into itself and develops the internal representation incrementally: In an even more recently published research (March 2017) from UC Berkeley have created astonishingly capable image to image translations using GANs and a novel kind of regularization. The major difficulty of training Deep Learning systems has been the lack of labeled data. So the next time you see some mind boggling Deep Learning results, seek to find the strange loops that are embedded in the method.

It just occurred to me, that after a couple of years tracking Deep Learning developments, that nobody has even bothered to create a map of what's going on! So I quickly decided to come up with a Deep Learning roadmap. A word of warning, this is just a partial map and doesn't cover the latest developments. Many of the ideas I write on this blog isn't even covered by this map. Anyway, here's a start of this and hope people start coming out of their labs to further expand on it.

Update: I have since refined these ideas in The Mythos of Model Interpretability, an academic paper presented at the 2016 ICML Workshop on Human Interpretability of Machine Learning. Neural networks, on the other hand, are black boxes. By this, it's suggested that we can pass input in, and observe what comes out, but we lack the ability to reason about what happened in the middle. To confirm the prevalence of this narrative, I ran a Google search for "neural network black box", yielding 2,410,000 results. By comparison, "logistic regression black box" turns up 600,000 results.

In honor of Mother's day, celebrated in many countries on Sunday, May 14, 2017, we revisit KDnuggets "Mother of All Data" cartoon. Big Data Predicted that 67.53% of you would remember! Here are other KDnuggets Big Data, Data Mining, and Data Science Cartoons. Recent KDnuggets Cartoons: Cartoon: Perfect Valentine's Dates Found With Data Analysis Cartoon: When Self-Driving Car Machine Learning takes you too far ... Cartoon: Thanksgiving, Big Data, and Turkey Data Science. Cartoon: Data Scientist - the sexiest job of the 21st century until ... Cartoon: Facebook data science experiments and Cats Cartoon: It all started with the iPhone answering my email Cartoon: Where humans are still ahead of Deep Learning Cartoon: A solution for Data Scientists allergies caused by Big Data Cartoon: Perfect Valentine's Dates Found With Data Analysis Cartoon: When Self-Driving Car Machine Learning takes you too far ... Cartoon: Thanksgiving, Big Data, and Turkey Data Science.

Machine learning refers to a type of Artificial Intelligence (AI) that allows computers to learn beyond their initial static programming. These new programs are developed to analyze patterns in past data sets in order to adapt. More advanced computer programs are even capable of altering their code in response to prior exposure to an unfamiliar set of inputs, which opens a whole set of possibilities for the future of AI. Some of the recent applications of machine learning include Google's self-driving car and the algorithm behind the success of its web search function, companies providing online recommendation offers based on user's' browsing history, and fraud detection. Deep learning is a branch of machine learning that focuses on the neural network model inspired by our understanding of the biology of the human brain.

This article explores in more detail the idea of Meta Learning that was previously introduced in a post "The Meta Model and Meta Meta Model of Deep Learning". In this post, I explore "Learning to Learn" as a Meta Learning approach. We have to be very careful to distinguish between Learning to Learn and Hyper Parameter Optimization (HPO). HPO and more generally searching for architectures differs from "learning to learn" in that that HPO explores the space of architectures while meta-learning explores the space of learning algorithms. Meta-learning is all the rage in research these days.

It's no secret that the U.S. healthcare system needs help. What's more surprising is the role AI can play in fixing it. In a talk at the GPU Technology Conference, Dr. Michael Dahlweid, chief medical officer of digital solutions for GE Healthcare, described the myriad problems in U.S. healthcare, and the scope for finding fixes with the expanded use of deep learning. The numbers can be depressing: About 100,000 people die needlessly every year, he said. Nearly a third of all healthcare spending -- more than $690 billion -- is money wasted.