In 1943, at the height of World War II, the U.S. military hired an audacious psychologist named B.F. Skinner to develop pigeon-guided missiles. These were the early days of munitions guidance technology, and the Allies were apparently quite desperate to find more reliable ways to get missiles to hit their targets. It went like this: Skinner trained pigeons to peck at an image of the military target projected onto a screen. Whenever their beaks hit the moving target dead center, he rewarded the birds with food pellets. Once the pigeons had learned how to peck at targets, they earned their wings: Skinner would strap three of his little pilots into a missile cockpit specially fitted with straps attached to gyroscopes that would steer the bomb. Now, when American jets released their pigeon-filled bombs, the birds would peck at an image of the bomb's target, their little straps twisting and bending, gyroscopes whirling, guiding the bomb and the birds to their final resting place. Used with permission of the artist. The military eventually pulled the plug on Project Pigeon, while Skinner continued to develop a discipline that came to be known as behavioral psychology. He just wanted to discover how to train animals (and his children) using scientific techniques of stimulus, reward, and punishment. Over the past three years, using techniques similar to those pioneered by Skinner, DeepMind has developed some of the most sophisticated machine-learning techniques in order to train a computer with artificial intelligence (AI) to master the ancient board game of Go. Weirdly enough, this millenia-old board game is the perfect demonstration of human complexity, machine limitations, and how powerful AI has become. For decades, researchers considered playing Go to be the holy grail of game-playing AI. No computer had ever come close to beating a professional in an even, full-board game. Intriguingly, AlphaGo plays Go with something akin to human-like intuition. Computers have always been good at doing the kinds of tasks that we can logically define, like multiplying large numbers, storing information, and playing recorded movies.

In Part 1, we looked at text, voice, and computer vision advancements in 2017. Reinforcement learning (RL), or learning with reinforcement, is one of the most interesting and actively developing approaches to machine learning. The essence of the approach is to learn the successful behavior of the agent in an environment that gives a reward through experience -- just as people learn throughout their lives. RL is actively used in games, robots, and system management (traffic, for example). Of course, everyone has heard about AlphaGo's victories in the game over the best professionals.

The development of model architectures and algorithms in the field of deep learning is largely constrained by the availability of efficient GPU implementations of elementary operations. One issue has been the lack of an efficient GPU implementation for sparse linear operations, which we're now releasing, together with initial results using them to implement a number of sparsity patterns. These initial results are promising but not definitive, and we invite the community to join us in pushing the limits of the architectures these kernels unlock. Dense layers (left) can be replaced with layers that are sparse and wide (center) or sparse and deep (right) while approximately retaining computation time. Sparse weight matrices, as opposed to dense weight matrices, have a large number of entries with a value of exactly zero.

Lunit Inc., a member company of the K-ICT Born2Global Centre, has developed a deep learning-based technology for analyzing medical images that dramatically lowers the rate of misdiagnosis. Currently, the AI technology is being subjected to a sophistication process in cooperation with major medical institutions in Korea, including Seoul National University Hospital, Severance Hospital of Yonsei University Health System, Samsung Medical Center, and Asan Medical Center. Anthony Paek, the CEO of Lunit, explained, "The data-driven imaging biomarker (DIB) technology that Lunit proposed for the first time ever in 2015 is an AI system that has learned abnormal and clinically significant image patterns from big data." He went on to add, "Currently, DIB technology has achieved an accuracy level comparable to that of human experts. In the future, however, we will have new DIB technologies capable of outperforming humans."

Speed to insight is going to emerge as the key competitive differentiator for businesses, as they start stepping into the era of compute-and-speed-hungry artificial intelligence(AI), and deep learning workloads. IBM, recently announced a new line of accelerated IBM Power Systems Servers, keeping this new requirement of businesses in mind. The systems are built on its new POWER9 processor, which reduces the training times of deep learning frameworks significantly from days to hours and allows building more accurate AI applications in considerably less time. "The era of AI demands a tremendous amount of processing power at unprecedented speed," said Monica Aggarwal, Vice President, IBM India Systems Development Lab. "To meet the demands of the cognitive workload, businesses need to change everything right from the start- the algorithms, the software, and the hardware as well. POWER9 systems bring an integrated AI platform designed to accelerate machine learning and deep learning with both software and hardware that are optimized to work together."

How smart are you when it comes to the nuances of Artificial Intelligence (AI)? When you read about the future of AI, it can seem like there are a lot of buzzwords being thrown around in the media. Differentiating among AI, machine learning and deep learning technologies can be confusing, especially when terms are being used interchangeably. Let's begin by clearing things up with a few definitions. First, there is AI, which refers to intelligence exhibited by machines in the form of human cognitive functions like visual perception, speech recognition, decision-making, and language translation.

In this paper, we present an initial attempt to learn evolution PDEs from data. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models. The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses. Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses. A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network. These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory). We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet). Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

However, the conventional deep neural network architecture is limited in learning representations for multi-task RL (MT-RL), as multiple tasks can refer to different kinds of representations. In this paper, we thus propose a novel deep neural network architecture, namely generalization tower network (GTN), which can achieve MT-RL within a single learned model. Specifically, the architecture of GTN is composed of both horizontal and vertical streams. In our GTN architecture, horizontal streams are used to learn representation shared in similar tasks. In contrast, the vertical streams are introduced to be more suitable for handling diverse tasks, which encodes hierarchical shared knowledge of these tasks. The effectiveness of the introduced vertical stream is validated by experimental results. Experimental results further verify that our GTN architecture is able to advance the state-of-the-art MT-RL, via being tested on 51 Atari games.

Background: Deep learning models are typically trained using stochastic gradient descent or one of its variants. These methods update the weights using their gradient, estimated from a small fraction of the training data. It has been observed that when using large batch sizes there is a persistent degradation in generalization performance - known as the "generalization gap" phenomena. Identifying the origin of this gap and closing it had remained an open problem. Contributions: We examine the initial high learning rate training phase. We find that the weight distance from its initialization grows logarithmically with the number of weight updates. We therefore propose a "random walk on random landscape" statistical model which is known to exhibit similar "ultra-slow" diffusion behavior. Following this hypothesis we conducted experiments to show empirically that the "generalization gap" stems from the relatively small number of updates rather than the batch size, and can be completely eliminated by adapting the training regime used. We further investigate different techniques to train models in the large-batch regime and present a novel algorithm named "Ghost Batch Normalization" which enables significant decrease in the generalization gap without increasing the number of updates. To validate our findings we conduct several additional experiments on MNIST, CIFAR-10, CIFAR-100 and ImageNet. Finally, we reassess common practices and beliefs concerning training of deep models and suggest they may not be optimal to achieve good generalization.

Researchers have also applied neural network-based approaches to cybersecurity tasks. Ryan et al. \shortciteryan1998intrusion train a standard neural network with one hidden layer to predict the probabilities that each of a set of ten users created a distribution of Unix commands for a given day. They detect a network intrusion when the probability is less than 0.5 for all ten users of the network. Differing from our work, their input features are not structured, and they do not train the network in an online fashion. Early work on modeling normal user activity on a network using RNNs was performed by Debar et al. \shortcitedebar1992neural. They train an RNN to convergence on a representative sequence of Unix command line arguments (from login to logout) and predict network intrusion when the trained network for that user does poorly at predicting the login to logout sequence.