This is all very promising. Before we can use these new sources of data to inform clinical decision-making, we need to overcome some significant technical challenges. While we have recently seen important advancements in machine learning and artificial intelligence, this work has been largely driven by consumer applications, with Google, IBM, Facebook, Microsoft, and Amazon leading the way. These companies have developed deep learning models that achieve near-human performance on certain tasks, even as their workings are largely incomprehensible to human users. Most people use the Amazon Echo without an understanding of the A.I. algorithms that underlie the speech recognition engine.

Thinc is the machine learning library powering spaCy. It features a battle-tested linear model designed for large sparse learning problems, and a flexible neural network model under development for spaCy v2.0. Thinc is a practical toolkit for implementing models that follow the "Embed, encode, attend, predict" architecture. It's designed to be easy to install, efficient for CPU usage and optimised for NLP and deep learning with text – in particular, hierarchically structured input and variable-length sequences. Thinc's deep learning functionality is still under active development: APIs are unstable, and we're not yet ready to provide usage support.

Deep learning models are complex and tricky to train, and I had a hunch that lack of model convergence/difficulties training probably explained the poor performance, not overfitting. We recreated python versions of the Leekasso and MLP used in the original post to the best of our ability, and the code is available here. The MLP used in the original analysis still looks pretty bad for small sample sizes, but our neural nets get essentially perfect accuracy for all sample sizes. A lot of parameters are problem specific (especially the parameters related to SGD) and poor choices will result in misleadingly bad performance.

Artificial intelligence is slowly proving that that video games aren't a total waste of time, at least for machines: It's through learning to play games that AI algorithms can acquire all sorts of generalizable skills, like problem-solving. Now, computer scientists from RWTH Aachen University in Germany and Microsoft Research have released the largest-ever database of human playthroughs for some of the most popular games for the Atari 2600. Artificial agents using deep learning techniques will be able to pull patterns out of these playthroughs and learn from them. According to a paper posted to the arXiv preprint server this week, which is undergoing peer review, the database contains more than 45 hours of gameplay from five games: Q*Bert, Ms. Pacman, Space Invaders, Video Pinball, and Montezuma's Revenge. Video games are an increasingly popular training ground for AI to solve general problems, like how to quickly arrive at a course of action, or how to effectively learn in an environment where the rewards for learning are sparse, which is notoriously the case for Montezuma's Revenge .

Deep generative models have achieved impressive success in recent years. Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as powerful frameworks for deep generative model learning, have largely been considered as two distinct paradigms and received extensive independent study respectively. This paper establishes formal connections between deep generative modeling approaches through a new formulation of GANs and VAEs. We show that GANs and VAEs are essentially minimizing KL divergences with opposite directions and reversed latent/visible treatments, extending the two learning phases of classic wake-sleep algorithm, respectively. The unified view provides a powerful tool to analyze a diverse set of existing model variants, and enables to exchange ideas across research lines in a principled way.

How does DeepMind choose research topics to focus on? Going through DeepMind's publication lists, their focus seems a bit more narrow than other labs. Yet they are probably one of the largest (if not the largest) research group with about 400 people (https://en.wikipedia.org/wiki/DeepMind) . For example, the have a great deal of work on generative models but no work on GANs. They also have a great deal of focus on Reinforcement Learning but not robotics.

Google announced expanded partnerships with three blue-chip academic medical centers this past week, where bioinformaticians will explore how its machine learning technology can be deployed in clinical settings to mine EHR data for improved outcomes. "Machine learning is mature enough to start accurately predicting medical events – such as whether patients will be hospitalized, how long they will stay, and whether their health is deteriorating despite treatment for conditions such as urinary tract infections, pneumonia, or heart failure," said Google Brain Team researcher Katherine Chou in a blog post. "Advanced machine learning can discover patterns in de-identified medical records to predict what is likely to happen next, and thus, anticipate the needs of the patients before they arise," she added. Google Brain is especially interested in putting machine learning to work predicting and preventing healthcare-associated infections, medication errors and hospital readmissions. As it does, the company will be "helping to harmonize the different ways data appears" among its partner hospitals, said Chou.

The king ate his lunch.

Robotic hands can play drums and even twirl objects with aplomb, but they're still poor at picking up unfamiliar objects. That's why UC Berkeley's DexNet 2.0 bot is so impressive -- using deep learning, it can successfully grasp random, real-world objects 99 percent of the time. What's more, the tech, developed with the help of Amazon, Google and Toyota, is far enough along that it could be put to work in manufacturing and supply chains in the near future. Researchers trained the DexNet 2.0 deep learning system using a vast library of 3D shapes and suitable grasp positions to match those objects. Using virtual, rather than real objects made it possible to train the AI much more quickly.

Over at Simply Stats Jeff Leek posted an article entitled "Don't use deep learning your data isn't that big" that I'll admit, rustled my jimmies a little bit. To be clear, I don't think deep learning is a universal panacea and I mostly agree with his central thesis (more on that later), but I think there are several things going on at once, and I'd like to explore a few of those further in this post. Jeff takes a look at the performance of two approaches to classify handwritten 0s vs. 1s from the well known MNIST data set. He compares the performance of a 5-layer neural net with hyperbolic tangent activations to the Leekasso, which just uses the 10 pixels with the smallest marginal p-values. He shows, perhaps surprisingly, that the Leekasso outperforms the neural net when you only have a dozen or so samples. Don't use deep learning if you have 100 samples because the model will overfit and you will get bad out of sample performance.