Doctors can prevent one of the most common causes of blindness in young children -- but only when they can detect it. The disease, called retinopathy of prematurity, or ROP, affects the youngest, smallest and most vulnerable infants. These are preterm babies born before 31 weeks who weigh less than 2¾ pounds. Doctors can treat ROP if they catch it early enough, but there's no objective way to determine which cases need treatment. Jayashree Kalpathy-Cramer thinks AI can make a difference.

Artificial Neural Networks are powerful function approximators capable of modelling solutions to a wide variety of problems, both supervised and unsupervised. As their size and expressivity increases, so too does the variance of the model, yielding a nearly ubiquitous overfitting problem. Although mitigated by a variety of model regularisation methods, the common cure is to seek large amounts of training data--which is not necessarily easily obtained--that sufficiently approximates the data distribution of the domain we wish to test on. In contrast, logic programming methods such as Inductive Logic Programming offer an extremely data-efficient process by which models can be trained to reason on symbolic domains. However, these methods are unable to deal with the variety of domains neural networks can be applied to: they are not robust to noise in or mislabelling of inputs, and perhaps more importantly, cannot be applied to non-symbolic domains where the data is ambiguous, such as operating on raw pixels. In this paper, we propose a Differentiable Inductive Logic framework, which can not only solve tasks which traditional ILP systems are suited for, but shows a robustness to noise and error in the training data which ILP cannot cope with. Furthermore, as it is trained by backpropagation against a likelihood objective, it can be hybridised by connecting it with neural networks over ambiguous data in order to be applied to domains which ILP cannot address, while providing data efficiency and generalisation beyond what neural networks on their own can achieve.

We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. A major approach to achieve this objective is to train a model that integrates all the information of different modalities into a joint representation and then to generate one modality from the corresponding other modality via this joint representation. We simply applied this approach to variational autoencoders (VAEs), which we call a joint multimodal variational autoencoder (JMVAE). However, we found that when this model attempts to generate a large dimensional modality missing at the input, the joint representation collapses and this modality cannot be generated successfully. Furthermore, we confirmed that this difficulty cannot be resolved even using a known solution. Therefore, in this study, we propose two models to prevent this difficulty: JMVAE-kl and JMVAE-h. Results of our experiments demonstrate that these methods can prevent the difficulty above and that they generate modalities bi-directionally with equal or higher likelihood than conventional VAE methods, which generate in only one direction. Moreover, we confirm that these methods can obtain the joint representation appropriately, so that they can generate various variations of modality by moving over the joint representation or changing the value of another modality.

Internet of things (IoT) applications have become increasingly popular in recent years, with applications ranging from building energy monitoring to personal health tracking and activity recognition. In order to leverage these data, automatic knowledge extraction - whereby we map from observations to interpretable states and transitions - must be done at scale. As such, we have seen many recent IoT data sets include annotations with a human expert specifying states, recorded as a set of boundaries and associated labels in a data sequence. These data can be used to build automatic labeling algorithms that produce labels as an expert would. Here, we refer to human-specified boundaries as breakpoints. Traditional changepoint detection methods only look for statistically-detectable boundaries that are defined as abrupt variations in the generative parameters of a data sequence. However, we observe that breakpoints occur on more subtle boundaries that are non-trivial to detect with these statistical methods. In this work, we propose a new unsupervised approach, based on deep learning, that outperforms existing techniques and learns the more subtle, breakpoint boundaries with a high accuracy. Through extensive experiments on various real-world data sets - including human-activity sensing data, speech signals, and electroencephalogram (EEG) activity traces - we demonstrate the effectiveness of our algorithm for practical applications. Furthermore, we show that our approach achieves significantly better performance than previous methods.

This resource is part of a series on specific topics related to data science: regression, clustering, neural networks, deep learning, decision trees, ensembles, correlation, Python, R, Tensorflow, SVM, data reduction, feature selection, experimental design, cross-validation, model fitting, and many more. To keep receiving these articles, sign up on DSC. Enjoy the reading, or become one of our bloggers and start posting articles and tutorials on DSC.

We covered Google's Cloud AutoML Vision last week and, as we predicted, Facebook has already come out with a platform for object detection of it's own – Detectron. Detectron is a software system developed by Facebook's AI Research team (FAIR) that "implements state-of the art object detection algorithms". It is written in Python and leverages the Caffee2 deep learning framework underneath. Detectron aims to provide a high quality and industry standard codebase for object detection research. The results it has posted are incredibly accurate.

In this post I will cover a few low rank tensor decomposition methods for taking layers in existing deep learning models and making them more compact. I will also share PyTorch code that uses Tensorly for performing CP decomposition and Tucker decomposition of convolutional layers. Although hopefully most of the post is self contained, a good review of tensor decompositions can be found here. The author of Tensorly also created some really nice notebooks about Tensors basics. That helped me getting started, and I recommend going through that.

Technology developed at the University of Waterloo is paving the way for artificial intelligence (AI) to break free of the internet and cloud computing. New deep-learning AI software produced with that technology is compact enough to fit on mobile computer chips for use in everything from smartphones to industrial robots. That would allow devices to operate independent of the internet while using AI that performs almost as well as tethered neural networks. "We feel this has enormous potential," said Alexander Wong, a systems design engineering professor and Waterloo and co-creator of the technology. "This could be an enabler in many fields where people are struggling to get deep-learning AI in an operational form."

Feeling rather good about myself as I'm writing this as I've just completed the first course of Andrew Ng's latest Deep Learning specialization on Coursera. I've been meaning to learn about Deep Learning for quite awhile now but haven't been able to wrap my heads around the theory aspect of it for longest of time. Previously, my foray into deep learning has been via Udacity's Deep Learning materials, random internet articles, and the Deep Learning textbook. Bought it from Amazon a few months ago, and am still going through the pages. Still finding it tough to find the time between going through a few pages, the day job, and sorting out the kids at night.

This is a PyTorch implementation of "Trust Region Policy Optimization (TRPO)". This is code mostly ported from original implementation by John Schulman. In contrast to another implementation of TRPO in PyTorch, this implementation uses exact Hessian-vector product instead of finite differences approximation. If you know how to make this code better, don't hesitate to send a pull request.