The popular name for Convolutional Neural Network (CNN) these days is "Deep Learning" (DL) or Deep Neural Network (DNN). Researchers at Google, Facebook, IBM, Nvidia, and potentially a number of other companies are experimenting with their own variations of DL algorithms based on CNN. Twitter recently acquired a company called Magic Pony to increase focus on DL. Applications for DL range from vision processing for self-driving car applications to natural language processing and face/object recognition, for potential applications in security, advertising, gaming, VR/AR, etc. A neural network is a computational algorithm that is loosely based on mechanics of the learning process in animals.

This time, Sebastian explains the difference between Deep Learning and "regular" machine learning. That's an interesting question, and I try to answer this is a very general way. The tl;dr version of this is: Deep learning is essentially a set of techniques that help we to parameterize deep neural network structures, neural networks with many, many layers and parameters. And if we are interested, a more concrete example: Let's start with multi-layer perceptrons (MLPs)… On a tangent: The term "perceptron" in MLPs may be a bit confusing since we don't really want only linear neurons in our network. Using MLPs, we want to learn complex functions to solve non-linear problems.

The purpose of this book is to help to spread TensorFlow knowledge among engineers who want to expand their wisdom in the exciting world of Machine Learning. We believe that anyone with an engineering background might require from now on Deep Learning, and Machine Learning in general, to apply it in their work. As the title indicates, it is a first contact with TensorFlow in order to get started with Deep Learning programming. The book has a practical nature, and therefore it reduces the theoretical part as much as possible, assuming that the reader has some basic understanding about Machine Learning.

Prediction tasks over nodes and edges in networks require careful effort in engineering features used by learning algorithms. Recent research in the broader field of representation learning has led to significant progress in automating prediction by learning the features themselves. However, present feature learning approaches are not expressive enough to capture the diversity of connectivity patterns observed in networks. Here we propose node2vec, an algorithmic framework for learning continuous feature representations for nodes in networks. In node2vec, we learn a mapping of nodes to a low-dimensional space of features that maximizes the likelihood of preserving network neighborhoods of nodes. We define a flexible notion of a node's network neighborhood and design a biased random walk procedure, which efficiently explores diverse neighborhoods. Our algorithm generalizes prior work which is based on rigid notions of network neighborhoods, and we argue that the added flexibility in exploring neighborhoods is the key to learning richer representations. We demonstrate the efficacy of node2vec over existing state-of-the-art techniques on multi-label classification and link prediction in several real-world networks from diverse domains. Taken together, our work represents a new way for efficiently learning state-of-the-art task-independent representations in complex networks.

A key goal of computer vision is to recover the underlying 3D structure from 2D observations of the world. In this paper we learn strong deep generative models of 3D structures, and recover these structures from 3D and 2D images via probabilistic inference. We demonstrate high-quality samples and report log-likelihoods on several datasets, including ShapeNet [2], and establish the first benchmarks in the literature. We also show how these models and their inference networks can be trained end-to-end from 2D images. This demonstrates for the first time the feasibility of learning to infer 3D representations of the world in a purely unsupervised manner.

The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable performance within an acceptable computational time. We presented a novel approach of optimising a deep neural network through regularisation of network architecture. We proposed regularisers which support a simple mechanism of dropping neurons during a network training process. The method supports the construction of a simpler deep neural networks with compatible performance with its simplified version. As a proof of concept, we evaluate the proposed method with examples including sparse linear regression, deep autoencoder and convolutional neural network. The valuations demonstrate excellent performance. The code for this work can be found in http://www.github.com/panweihit/

As the automation of physical and knowledge work advances, many jobs will be redefined rather than eliminated--at least in the short term. The potential of artificial intelligence and advanced robotics to perform tasks once reserved for humans is no longer reserved for spectacular demonstrations by the likes of IBM's Watson, Rethink Robotics' Baxter, DeepMind, or Google's driverless car. Just head to an airport: automated check-in kiosks now dominate many airlines' ticketing areas. Pilots actively steer aircraft for just three to seven minutes of many flights, with autopilot guiding the rest of the journey. Passport-control processes at some airports can place more emphasis on scanning document bar codes than on observing incoming passengers.

Computer vision engineers have used machine learning techniques for decades to detect objects of interest in images and to classify or identify categories of objects. They extract features representing points, regions, or objects of interest and then use those features to train a model to classify or learn patterns in the image data. In traditional machine learning, feature selection is a time-consuming manual process. Feature extraction usually involves processing each image with one or more image processing operations, such as calculating gradient to extract the discriminative information from each image. Deep learning algorithms can learn features, representations, and tasks directly from images, text, and sound, eliminating the need for manual feature selection.

I guess it depends on what kind of project you are doing, but for something like scientific computing the issue is development time, not compile time. Integrating and using code is one of the main bottlenecks. Headers make this really easy. If you do scientific computing and need to use C, then you are probably doing a process or memory demanding task where compile time is non-existent in comparison.

This article is an interview with computational linguist Jason Baldridge. It's for anyone who's interested in, or needs to know about, natural language processing (NLP). Jason and NLP go way back. He joined the University of Texas linguistics faculty in 2005 and, a few years back, helped build a text-analytics system for social-media agency Converseon. Jason's Austin start-up, People Pattern, applies NLP and machine learning for social-audience insights; he co-founded the company in 2013 and serves as chief scientist.