Rely on the Intel Nervana AI Academy to help you increase your knowledge base and put machine learning to use quickly, efficiently, and cost-effectively on Intel architecture. In this webinar, we describe various deep learning uses and highlight those in which Caffe* was used, and describe how Caffe is optimized for Intel architecture. In this webinar, we continue our exploration of deep learning topics including multilayer perceptron, convolutional neural networks, recurrent neural networks, cost functions, and back propogation. Learn how tools, libraries, and Intel platforms are co-optimized for performance and inference - to classify, recognize, and process new inputs. See practical examples and discover new opportunities to apply AI in the real world.

This post highlights a number of important applications found for deep learning so far. It is well known that 80% of data is unstructured. Unstructured data is the messy stuff every quantitative analyst tries to traditionally stay away from. It can include images of accidents, text notes of loss adjusters, social media comments, claim documents and review of medical doctors etc. Unstructured data has massive potential but has never been traditionally considered as a source of insight before. Deep Learning is becoming the method of choice for its exceptional accuracy and capturing capacity for unstructured data.

The remarkable success of our GPU Technology Conference this month demonstrated to anyone still in doubt the extraordinary momentum of the AI revolution. Throughout the four-day event here in Silicon Valley, attendees from the world's leading companies in media and entertainment, manufacturing, healthcare and transportation shared stories of their breakthroughs made possible by GPU computing. The numbers tell a powerful story. With more than 7,000 attendees, 150 exhibitors and 600 technical sessions, our eighth annual GTC was our largest yet. The world's top 15 tech companies were there, as were the world's top 10 automakers, and more than 100 startups focusing on AI and VR.

There is also an aspect of deep learning models that I see gets sort of lost in translation when coming from other fields of machine learning. Most tutorials and introductory material to deep learning describe these models as composed by hierarchically-connected layers of nodes where the first layer is the input and the last layer is the output and that you can train them using some form of stochastic gradient descent. After maybe some brief mentions on how stochastic gradient descent works and what backpropagation is, the bulk of the explanation focuses on the rich landscape of neural network types (convolutional, recurrent, etc.). The optimization methods themselves receive little additional attention, which is unfortunate since it's likely that a big (if not the biggest) part of why deep learning works is because of those particular methods (check out, e.g. this post from Ferenc Huszár's and this paper taken from that post), and knowing how to optimize their parameters and how to partition data to use them effectively is crucial to get good convergence in a reasonable amount of time. Exactly why stochastic gradients matter so much is still unknown, but some clues are emerging here and there.

In an exciting demonstration of the power of artificial intelligence (AI) and the diversity of species, a team composed of two programmers and an ornithologist (an expert on birds) created a map of visualized bird sounds. Coders Manny Tan and Kyle McDonald worked with ornithologist Jessie Barry to create this visually euphonious interactive map of bird sounds. Tan and McDonald used machine learning to organize thousands of bird sounds from a collection by Cornell University. They didn't supply their algorithm with tags or even names of the bird sounds. Instead, they wanted to see how it would learn to organize all the data by listening to the bird sounds.

Data science experts from Netflix, Riot Games, Verizon Wireless, and Quora shared best practices during the opening panel session at DataScience: Elevate.

Instant Visual Translation Example of instant visual translation, taken from the Google Blog. A more complex variation of this task called object detection involves specifically identifying one or more objects within the scene of the photograph and drawing a box around them. In 2014, there were an explosion of deep learning algorithms achieving very impressive results on this problem, leveraging the work from top models for object classification and object detection in photographs. Generally, the systems involve the use of very large convolutional neural networks for the object detection in the photographs and then a recurrent neural network like an LSTM to turn the labels into a coherent sentence.

The vanishing gradient problem was a major obstacle for the success of deep learning. In recent years it was gradually alleviated through multiple different techniques. However the problem was not really overcome in a fundamental way, since it is inherent to neural networks with activation functions based on dot products. In a series of papers, we are going to analyze alternative neural network structures which are not based on dot products. In this first paper, we revisit neural networks built up of layers based on distance measures and Gaussian activation functions. These kinds of networks were only sparsely used in the past since they are hard to train when using plain stochastic gradient descent methods. We show that by using Root Mean Square Propagation (RMSProp) it is possible to efficiently learn multi-layer neural networks. Furthermore we show that when appropriately initialized these kinds of neural networks suffer much less from the vanishing and exploding gradient problem than traditional neural networks even for deep networks.

Learning automatically the structure of object categories remains an important open problem in computer vision. In this paper, we propose a novel unsupervised approach that can discover and learn landmarks in object categories, thus characterizing their structure. Our approach is based on factorizing image deformations, as induced by a viewpoint change or an object deformation, by learning a deep neural network that detects landmarks consistently with such visual effects. Furthermore, we show that the learned landmarks establish meaningful correspondences between different object instances in a category without having to impose this requirement explicitly. We assess the method qualitatively on a variety of object types, natural and man-made. We also show that our unsupervised landmarks are highly predictive of manually-annotated landmarks in face benchmark datasets, and can be used to regress these with a high degree of accuracy.

We consider the problem of efficient "on the fly" tuning of existing, or {\it legacy}, Artificial Intelligence (AI) systems. The legacy AI systems are allowed to be of arbitrary class, albeit the data they are using for computing interim or final decision responses should posses an underlying structure of a high-dimensional topological real vector space. The tuning method that we propose enables dealing with errors without the need to re-train the system. Instead of re-training a simple cascade of perceptron nodes is added to the legacy system. The added cascade modulates the AI legacy system's decisions. If applied repeatedly, the process results in a network of modulating rules "dressing up" and improving performance of existing AI systems. Mathematical rationale behind the method is based on the fundamental property of measure concentration in high dimensional spaces. The method is illustrated with an example of fine-tuning a deep convolutional network that has been pre-trained to detect pedestrians in images.