To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed. Dropout variational inference (VI) for example has been used for machine vision and medical applications, but VI can severely underestimates model uncertainty. Alpha-divergences are alternative divergences to VI's KL objective, which are able to avoid VI's uncertainty underestimation. But these are hard to use in practice: existing techniques can only use Gaussian approximating distributions, and require existing models to be changed radically, thus are of limited use for practitioners. We propose a re-parametrisation of the alpha-divergence objectives, deriving a simple inference technique which, together with dropout, can be easily implemented with existing models by simply changing the loss of the model. We demonstrate improved uncertainty estimates and accuracy compared to VI in dropout networks. We study our model's epistemic uncertainty far away from the data using adversarial images, showing that these can be distinguished from non-adversarial images by examining our model's uncertainty.

Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) methods generally rely on being able to learn and update models from small amounts of data. Recent advances in deep learning, on the other hand, are notorious for their dependence on large amounts of data. Second, many AL acquisition functions rely on model uncertainty, yet deep learning methods rarely represent such model uncertainty. In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way. We develop an active learning framework for high dimensional data, a task which has been extremely challenging so far, with very sparse existing literature. Taking advantage of specialised models such as Bayesian convolutional neural networks, we demonstrate our active learning techniques with image data, obtaining a significant improvement on existing active learning approaches. We demonstrate this on both the MNIST dataset, as well as for skin cancer diagnosis from lesion images (ISIC2016 task).

The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describe relationships among instances. In this work we introduce an approach for relational unsupervised representation learning. Viewing a relational dataset as a hypergraph, new features are obtained by clustering vertices and hyperedges. To find a representation suited for many relational learning tasks, a wide range of similarities between relational objects is considered, e.g. feature and structural similarities. We experimentally evaluate the proposed approach and show that models learned on such latent representations perform better, have lower complexity, and outperform the existing approaches on classification tasks.

ChainerRL is a deep reinforcement learning library that implements various state-of-the-art deep reinforcement algorithms in Python using Chainer, a flexible deep learning framework. ChainerRL is tested with Python 2.7 and 3.5.1 . For other requirements, see requirements.txt. ChainerRL contains atari_py as dependencies, and windows users may face error while installing it. This problem is discussed in OpenAI gym issues, and one possible counter measure is to enable "Bash on Ubuntu on Windows" for Windows 10 users.

In March 2016 Insilico Medicine initiated a research collaboration with Life Extension to apply advanced bioinformatic methods and deep learning algorithms to screen for naturally occurring compounds that may slow down or even reverse the cellular and molecular mechanisms of aging. Today Life Extension (LE) launched a new line of nutraceuticals called GEROPROTECTTM, and the first product in the series called Ageless CellTM combines some of the natural compounds that were shortlisted by Insilico Medicine's algorithms and are generally recognized as safe (GRAS). "Life Extension's mission is to extend the healthy human lifespan; and as such, we are focused on identifying natural products with critical health and wellness properties," said Andrew G. Swick, PhD, senior vice president of scientific affairs, discovery research and product development for Life Extension. "Our collaboration with Insilico Medicine fostered a novel approach to formulating anti-aging supplements utilizing artificial intelligence and sophisticated biologically-inspired algorithms and resulted in the very first AI formulated supplement," Swick said. The global nutraceuticals market was valued at US$165.62 billion in 2014 by Transparency Market Research and is expected to reach US$278.96 billion by 2021.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

The Chinese government, who recently announced they want to build AI based cruise missiles and a new nationwide Social Credit Scoring system, has approved a plan to create a next generation artificial intelligence (AI) national laboratory network which is expected to help China close the gap with its Western counterparts, many of whom now seem to be driving at full speed towards a world where artificial intelligence is the norm, not the exception. The National Development and Reform Commission (NDRC) approved plans for a national engineering lab to support the research and development of deep learning technologies last week, but in a twist the lab will be online only, and won't have a physical presence. The NDRC commissioned Baidu, the Chinese search engine giant, to create the lab in collaboration with Tsinghua University and Peking University, as well as the China Academy of Information and Communications Technology, and the China Electronics Standardization Institute. The project will be led by Baidu's deep learning institute chief Lin Yuanqing and scientist Xu Wei, along with academics from the Chinese Academy of Sciences, Zhang Bo and Li Wei. Baidu, who by all accounts are rapidly becoming the Google of China, certainly with respect to AI, will also provide the deep learning computing, algorithms and big data for the project. The lab will focus on seven different areas of the field: deep learning, computer vision and sensing, computer listening, biometric identification, new forms of human-computer interaction, standardized services, and deep learning intellectual property rights.

For those of you who have an interest, and or involvement in "Deep Learning" or want to learn more I've created this matrix. It's by no means all inclusive. It will provide you with a landscape of some Deep Learning resources to get you started or complement resources you might already have. The original version is available here as a 5-page PDF document. You can click on the 5 images below to zoom in.

This is programming assignment 3 from the course T81-855: Applications of Deep Learning at Washington University in St. Louis. All students must create a Kaggle account and submit a solution. Once you have submitted your solution entry log into Blackboard (at WUSTL) and submit a single file telling me your Kaggle name on the leaderboard (you do not need to register to Kaggle with your real name). This competition will be visible to the public, so there may be non-student submissions as well as student. The data set for this competition consists of 7 input columns that should be used to predict an outcome.

While participating in Jeremy Howard's excellent deep learning course I realized I was a little rusty on the prerequisites and my fuzziness was impacting my ability to understand concepts like backpropagation. I decided to put together a few wiki pages on these topics to improve my understanding. Here is a prettier version of my linear algebra page. In the context of deep learning, linear algebra is a mathematical toolbox that offers helpful techniques for manipulating groups of numbers simultaneously. It provides structures like vectors and matrices (spreadsheets) to hold these numbers and new rules for how to add, subtract, multiply, or divide them.