This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, and contrastive divergence in a unified probabilistic formulation. This unification makes the processes of sampling, inference, energy evaluation readily available without the need for costly Markov chain Monte Carlo methods. Our experiments demonstrate that the divergence triangle is capable of learning (1) an energy-based model with well-formed energy landscape, (2) direct sampling in the form of a generator network, and (3) feed-forward inference that faithfully reconstructs observed as well as synthesized data. The divergence triangle is a robust training method that can learn from incomplete data.

Deep neural networks are over-parameterized, which implies that the number of parameters are much larger than the number of samples used to train the network. Even in such a regime deep architectures do not overfit. This phenomenon is an active area of research and many theories have been proposed trying to understand this peculiar observation. These include the Vapnik Chervonenkis (VC) dimension bounds and Rademacher complexity bounds which show that the capacity of the network is characterized by the norm of weights rather than the number of parameters. However, the effect of input noise on these measures for shallow and deep architectures has not been studied. In this paper, we analyze the effects of various regularization schemes on the complexity of a neural network which we characterize with the loss, $L_2$ norm of the weights, Rademacher complexities (Directly Approximately Regularizing Complexity-DARC1), VC dimension based Low Complexity Neural Network (LCNN) when subject to varying degrees of Gaussian input noise. We show that $L_2$ regularization leads to a simpler hypothesis class and better generalization followed by DARC1 regularizer, both for shallow as well as deeper architectures. Jacobian regularizer works well for shallow architectures with high level of input noises. Spectral normalization attains highest test set accuracies both for shallow and deeper architectures. We also show that Dropout alone does not perform well in presence of input noise. Finally, we show that deeper architectures are robust to input noise as opposed to their shallow counterparts.

Conversational agents have begun to rise both in the academic (in terms of research) and commercial (in terms of applications) world. This paper investigates the task of building a non-goal driven conversational agent, using neural network generative models and analyzes how the conversation context is handled. It compares a simpler Encoder-Decoder with a Hierarchical Recurrent Encoder-Decoder architecture, which includes an additional module to model the context of the conversation using previous utterances information. We found that the hierarchical model was able to extract relevant context information and include them in the generation of the output. However, it performed worse (35-40%) than the simple Encoder-Decoder model regarding both grammatically correct output and meaningful response. Despite these results, experiments demonstrate how conversations about similar topics appear close to each other in the context space due to the increased frequency of specific topic-related words, thus leaving promising directions for future research and how the context of a conversation can be exploited.

By applying deep-learning techniques to a set of phase-contrast microscopy images, Japanese researchers have been able to identify the nature and origin of different cancer cells with 96% accuracy. This approach could lead to better cancer treatments (Cancer Res. The researchers, from Osaka University, used a convolutional neural network (CNN), a common scheme used in deep learning, to analyse the images. CNNs work by applying to the input image a set of connected filters and mathematical functions that, similarly to neurons, can be trained to extract specific features. In medical imaging, CNNs are modelled on the human visual system, with low layers that capture fine details such as edges, and higher levels that capture complex features reflecting the whole image.

Intel announced Nauta, an open source deep learning project based on Kubernetes. The project comes with select open source components and Intel-developed custom applications, tools, and scripts for building deep learning models. According to Intel, Nauta provides a multi-user, distributed computing environment for running deep learning model training experiments on systems based on Intel Xeon processor. The software foundation for the distributed platform is built on Kubernetes – industry's leading container orchestration engine. Mainstream deep learning tools and frameworks such as TensorFlow, TensorBoard and Jupyter Hub are tightly integrated with the platform.

The level of expectation surrounding AI in healthcare has reached fever pitch in recent years, with a number of pilot projects achieving positive early results. Most of these projects involved AI systems being trained on a sample dataset of medical data, such as x-rays or other medical imagery, after which the system was capable of providing early detection of various conditions. The challenge for many of these systems is that they were usually trained on data from a single healthcare provider, with a common health IT system. A recent study highlights how when faced with data from different health systems, such AI technologies often perform much worse than doctors. The research, conducted by the Icahn School of Medicine at Mount Sinai, used convolutional neural networks (CNNs) to analyze a bunch of chest X-ray images with the aim of diagnosing pneumonia.

Despite dazzling advances in AI, robots are still horribly ham-fisted. Increasingly, researchers and companies are turning to machine learning to make them more adaptive and dexterous. This typically means feeding the robot a video of what's in front of it and asking it to work out how it should move in order to manipulate that object. For instance, researchers at OpenAI, a nonprofit in San Francisco, taught a robotic hand to manipulate a child's block in this way. By signing up you agree to receive email newsletters and notifications from MIT Technology Review.

In the deep learning inferencing game, there are plenty of chipmakers, large and small, developing custom-built ASICs aimed at this application set. But one obscure company appears to have beat them to the punch. Habana Labs, a fabless semiconductor startup, began sampling its purpose-built inference processor for select customers back in September 2018, coinciding with the company's emergence from stealth mode. Eitan Medina, Habana's Chief Business Officer, claims its HL-1000 chip is now "the industry's highest performance inference processor." It's being offered to customers in a PCIe card that goes by name of Goya.

Ever since convolutional neural networks began outperforming humans in specific image recognition tasks, research in the field of computer vision has proceeded at breakneck pace. The basic architecture of CNNs (or ConvNets) was developed in the 1980s. Yann LeCun improved upon the original design in 1989 by using backpropagation to train models to recognize handwritten digits. We've come a long way since then. In 2018, we saw novel architecture designs that improve upon performance benchmarks and also expand the range of media that machine learning models can analyze. We also saw a number of breakthroughs with media generation which enable photorealistic style transfer, high-resolution image generation, and video-to-video synthesis.

Inside this tutorial, you will learn how to configure macOS Mojave for deep learning. After you've gone through this tutorial, your macOS Mojave system will be ready for (1) deep learning with Keras and TensorFlow, and (2) ready for Deep Learning for Computer Vision with Python. A tutorial on configuring Mojave has been a long time coming on my blog since the Mojave OS was officially released in September 2018. The OS was plagued with problems from the get-go, and I decided to hold off. I'm still actually running High Sierra on my machines, but after putting this guide together I feel confident in recommending Mojave to PyImageSearch readers. Apple has fixed most of the bugs, but as you'll see in this guide, Homebrew (an unofficial package manager for macOS) doesn't make everything especially easy.