Before TensorFlow, PyTorch and Caffe; Theano was the major library for deep learning development. However, the library's development and support will end after the upcoming Theano 1.0 release. The news came in an email from Theano's main developer Pascal Lamblin and Yoshua Bengio, notable expert on artificial neural networks and deep learning. "We will continue minimal maintenance to keep it working for one year, but we will stop actively implementing new features. Supporting Theano is no longer the best way we can enable the emergence and application of novel research ideas," wrote Prof. Bengio.

This is part three of our building a deep learning machine series. In this post I will outline how to install the drivers and packages needed to get up and running with TensorFlow's deep learning framework. To start, install Ubuntu 14.04 Server. The download link is here. If you are using AWS, or some other cloud virtual machine provider, simply create an instance with Ubuntu 14.04 and ssh into the machine.

Deep learning has emerged in the last few years as a premier technology for building intelligent systems that learn from data. Deep neural networks, originally roughly inspired by how the human brain learns, are trained with large amounts of data to solve complex tasks with unprecedented accuracy. With open source frameworks making this technology widely available, it is becoming a must-know for anybody involved with big data and machine learning. TensorFlow is currently the leading open source software for deep learning, used by a rapidly growing number of practitioners working on computer vision, natural language processing (NLP), speech recognition, and general predictive analytics. This book is an end-to-end guide to TensorFlow designed for data scientists, engineers, students, and researchers.

In this course, you will learn the foundations of deep learning. When you finish this class, you will: – – This course also teaches you how Deep Learning actually works, rather than presenting only a cursory or surface-level description. This is the first course of the Deep Learning Specialization. To help make deep learning even more accessible to engineers and data scientists at large, Google has launched a free Deep Learning Course. This short, intensive course provides you with all the basic tools and vocabulary to get started with deep learning, and walks you through how to use it to address some of the most common machine learning problems.

Deep learning has had great success in computer vision and other artificial intelligence tasks [1]. Underlying this success is a new way to approximate functions, from an additive construction commonly used in approximation theory to a compositional construction used in deep neural networks. The compositional construction seems to be particularly powerful in high dimensions. This suggests that deep neural network based models can be of use in other contexts that involve constructing functions. This includes solving partial differential equations, molecular modeling, model reduction, etc. These aspects have been explored recently in [2, 3, 4, 5, 6, 7]. 1 In this paper, we continue this line of work and propose a new algorithm for solving variational problems. We call this new algorithm the Deep Ritz method since it is based on using the neural network representation of functions in the context of the Ritz method. The Deep Ritz method has a number of interesting and promising features, which we explore later in the paper.

Data augmentation is a ubiquitous technique for increasing the size of labeled training sets by leveraging task-specific data transformations that preserve class labels. While it is often easy for domain experts to specify individual transformations, constructing and tuning the more sophisticated compositions typically needed to achieve state-of-the-art results is a time-consuming manual task in practice. We propose a method for automating this process by learning a generative sequence model over user-specified transformation functions using a generative adversarial approach. Our method can make use of arbitrary, non-deterministic transformation functions, is robust to misspecified user input, and is trained on unlabeled data. The learned transformation model can then be used to perform data augmentation for any end discriminative model. In our experiments, we show the efficacy of our approach on both image and text datasets, achieving improvements of 4.0 accuracy points on CIFAR-10, 1.4 F1 points on the ACE relation extraction task, and 3.4 accuracy points when using domain-specific transformation operations on a medical imaging dataset as compared to standard heuristic augmentation approaches.

Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the entire distribution over sensor readings. Discriminative models do not suffer from this limitation, but are typically more complex to train as latent variable models for state estimation. We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks. Our model can be viewed as a type of recurrent neural network, and the connection to probabilistic filtering allows us to design a network architecture that is particularly well suited for state estimation. We evaluate our approach on synthetic tracking task with raw image inputs and on the visual odometry task in the KITTI dataset. The results show significant improvement over both standard generative approaches and regular recurrent neural networks.

Scientists have made amazing advances enabling machines to understand language and process images for such applications as facial recognition, image classification (e.g., "cat" or "dog") and translation of texts. Work in the IBM Research lab in Dublin this summer was focused on a very different problem: using AI techniques such as deep learning to forecast a physical process, namely, ocean waves. Traditional physics-based models are driven by external forces: The tides rise and fall, winds blow in different directions, the depth and physical properties of water influence the speed and height of the waves. These physical processes and their relationships are encapsulated in the differential equations that are coded into numerical models of wave transport. The nature of the computations typically demands High Performance Computing infrastructure to resolve the equations.

This course is all about the application of deep learning and neural networks to reinforcement learning. If you've taken my first reinforcement learning class, then you know that reinforcement learning is on the bleeding edge of what we can do with AI. Specifically, the combination of deep learning with reinforcement learning has led to AlphaGo beating a world champion in the strategy game Go, it has led to self-driving cars, and it has led to machines that can play video games at a superhuman level. Reinforcement learning has been around since the 70s but none of this has been possible until now. The world is changing at a very fast pace. The state of California is changing their regulations so that self-driving car companies can test their cars without a human in the car to supervise.

DeepSchool.io is an open-source, community based project to teach the A-Z of Deep Learning (DL). All lessons are interactive and (hopefully) funny, occasionally taking jabs at Mr. Trump (check out this notebook on Trump Tweets). This project came out of a weekly class that I did at Arbor Networks where I work as a Data Scientist. Personally I come from a background where I did a PhD in Machine Learning. However, with the development of tools such as Keras, DL has become a lot more accessible to the general community.