We are in the golden age of machine learning. From Microsoft's new computer vision system outperforming humans to Google's AI algorithm mastering the ancient game of Go, scientists have already achieved what many thought would take years to accomplish. There is growing excitement about what new applications deep learning will enable next. Will we soon rely on computers to keep us safe on our daily commute in self-driving cars? Will we use machines to diagnose us based on our symptoms and medical history?

The term'singularity' has many meanings. In theoretical physics, singularity refers to a one dimensional point in space where the gravitational field becomes infinity. An example of this would be at the centre of a black hole, where the laws of physics cease to hold true. Technological singularity on the other hand, a term first attributed to John Von Neumann in the 1950s, refers to the hypothesis that machine intelligence will one day exceed human intellect. What this means is that, someday, Artificial Intelligence (AI) that constantly improves on its own can surpass human intelligence.

Not available

As deep learning is gaining in popularity, creative applications are gaining traction as well. Looking at music generation through deep learning, new algorithms and songs are popping up on a weekly basis. In this post we will go over six major players in the field, and point out some difficult challenges these systems still face. GitHub links are provided for those who are interested in the technical details (or if you're looking to generate some music of your own). Magenta is Google's open source deep learning music project.

At the core of the AI system are two different neural networks -- a vision network and an imitation network. These two work behind the scenes to provide the remarkable capability to imitate human actions, a giant step closer to building true AI systems. A robotic arm repeats the process of picking up blocks and stacking them in a particular configuration. It does this by witnessing just once a simulated demonstration performed by a human using a VR headset. Researchers have used thousands of simulated images to train the vision network.

Elon Musk keeps surprising the world with his technological breakthroughs. OpenAI, a non-profit company focused on AI research, recently made an announcement regarding its groundbreaking AI invention. It has developed an AI system that can complete an actual physical task after watching just one demonstration of the task. At the core of the AI system are two different neural networks -- a vision network and an imitation network. These two work behind the scenes to provide the remarkable capability to imitate human actions, a giant step closer to building true AI systems.

We describe a fully data driven model that learns to perform a retrosynthetic reaction prediction task, which is treated as a sequence-to-sequence mapping problem. The end-to-end trained model has an encoder-decoder architecture that consists of two recurrent neural networks, which has previously shown great success in solving other sequence-to-sequence prediction tasks such as machine translation. The model is trained on 50,000 experimental reaction examples from the United States patent literature, which span 10 broad reaction types that are commonly used by medicinal chemists. We find that our model performs comparably with a rule-based expert system baseline model, and also overcomes certain limitations associated with rule-based expert systems and with any machine learning approach that contains a rule-based expert system component. Our model provides an important first step towards solving the challenging problem of computational retrosynthetic analysis.

It is challenging to develop stochastic gradient based scalable inference for deep discrete latent variable models (LVMs), due to the difficulties in not only computing the gradients, but also adapting the step sizes to different latent factors and hidden layers. For the Poisson gamma belief network (PGBN), a recently proposed deep discrete LVM, we derive an alternative representation that is referred to as deep latent Dirichlet allocation (DLDA). Exploiting data augmentation and marginalization techniques, we derive a block-diagonal Fisher information matrix and its inverse for the simplex-constrained global model parameters of DLDA. Exploiting that Fisher information matrix with stochastic gradient MCMC, we present topic-layer-adaptive stochastic gradient Riemannian (TLASGR) MCMC that jointly learns simplex-constrained global parameters across all layers and topics, with topic and layer specific learning rates. State-of-the-art results are demonstrated on big data sets.

Deep neural networks require a large amount of labeled training data during supervised learning. However, collecting and labeling so much data might be infeasible in many cases. In this paper, we introduce a source-target selective joint fine-tuning scheme for improving the performance of deep learning tasks with insufficient training data. In this scheme, a target learning task with insufficient training data is carried out simultaneously with another source learning task with abundant training data. However, the source learning task does not use all existing training data. Our core idea is to identify and use a subset of training images from the original source learning task whose low-level characteristics are similar to those from the target learning task, and jointly fine-tune shared convolutional layers for both tasks. Specifically, we compute descriptors from linear or nonlinear filter bank responses on training images from both tasks, and use such descriptors to search for a desired subset of training samples for the source learning task. Experiments demonstrate that our selective joint fine-tuning scheme achieves state-of-the-art performance on multiple visual classification tasks with insufficient training data for deep learning. Such tasks include Caltech 256, MIT Indoor 67, Oxford Flowers 102 and Stanford Dogs 120. In comparison to fine-tuning without a source domain, the proposed method can improve the classification accuracy by 2% - 10% using a single model.

By analyzing CT scans from 48 patients, the deep learning algorithms could predict whether they'd die within five years with 69 percent accuracy -- "broadly similar" to scores from human diagnosticians, the paper says. "Instead of focusing on diagnosing diseases, the automated systems can predict medical outcomes in a way that doctors are not trained to do, by incorporating large volumes of data and detecting subtle patterns." For this study, the system was looking for things like emphysema, an enlarged heart and vascular conditions like blood clotting.The deep learning system was trained to analyze over 16,000 image features that could indicate signs of disease in those organs. The goal was not to build a grim diagnostic system, and the AI only analyzed retrospective patient data.