While deep learning models and techniques have achieved great empirical success, our understanding of the source of success in many aspects remains very limited. In an attempt to bridge the gap, we investigate the decision boundary of a production deep learning architecture with weak assumptions on both the training data and the model. We demonstrate, both theoretically and empirically, that the last weight layer of a neural network converges to a linear SVM trained on the output of the last hidden layer, for both the binary case and the multi-class case with the commonly used cross-entropy loss. Furthermore, we show empirically that training a neural network as a whole, instead of only fine-tuning the last weight layer, may result in better bias constant for the last weight layer, which is important for generalization. In addition to facilitating the understanding of deep learning, our result can be helpful for solving a broad range of practical problems of deep learning, such as catastrophic forgetting and adversarial attacking. The experiment codes are available at https://github.com/lykaust15/NN_decision_boundary

For example, given a view of a chair with two rear legs occluded by front legs, humans are easily able to guess the most likely shape behind the visible parts. Recent advances in deep neural networks and data driven approaches show promising results in dealing with such a task. In this paper, we aim to acquire the complete and highresolution 3D shape of an object given a single depth view. By leveraging the high performance of 3D convolutional neural nets and large open datasets of 3D models, our approach learns a smooth function that maps a 2.5D view to a complete and dense 3D shape. In particular, we train an endto-end model which estimates full volumetric occupancy from a single 2.5D depth view of an object. As a result, the learnt 3D structure tends to be coarse and inaccurate. In order to generate higher resolution 3D objects with efficient computation, Octree representation has been recently introduced in [13] [14] [15]. However, increasing the density of output 3D shapes would also inevitably pose a great challenge to learn the geometric details for high resolution 3D structures, which has yet to be explored. Recently, deep generative models achieve impressive success in modeling complex high-dimensional data distributions, among which Generative Adversarial Networks (GANs) [16] and Variational Autoencoders (VAEs) [17] emerge as two powerful frameworks for generative learning, including image and text generation [18] [19], and latent space learning [20] [21]. In the past few years, a number of works [22] [23] [24] [25] applied such generative models to learn latent space to represent 3D object shapes, in order to solve tasks such as new image generation, object classification, recognition and shape retrieval. Abstract--In this paper, we propose a novel approach, 3D-RecGAN, which reconstructs the complete 3D structure of a given object from a single arbitrary depth view using generative adversarial networks.

This paper addresses the problem of mapping natural language text to knowledge base entities. The mapping process is approached as a composition of a phrase or a sentence into a point in a multi-dimensional entity space obtained from a knowledge graph. The compositional model is an LSTM equipped with a dynamic disambiguation mechanism on the input word embeddings (a Multi-Sense LSTM), addressing polysemy issues. Further, the knowledge base space is prepared by collecting random walks from a graph enhanced with textual features, which act as a set of semantic bridges between text and knowledge base entities. The ideas of this work are demonstrated on large-scale text-to-entity mapping and entity classification tasks, with state of the art results.

Designing shared neural architecture plays an important role in multi-task learning. The challenge is that finding an optimal sharing scheme relies heavily on the expert knowledge and is not scalable to a large number of diverse tasks. Inspired by the promising work of neural architecture search (NAS), we apply reinforcement learning to automatically find possible shared architecture for multi-task learning. Specifically, we use a controller to select from a set of shareable modules and assemble a task-specific architecture, and repeat the same procedure for other tasks. The controller is trained with reinforcement learning to maximize the expected accuracies for all tasks. We conduct extensive experiments on two types of tasks, text classification and sequence labeling, which demonstrate the benefits of our approach.

Whether you are new to deep learning or a seasoned veteran, setting up an environment for training a neural network can be painful sometimes. Would it be awesome to make training a neural network as simple as loading a webpage followed by a few clicks and you are ready to make inference with it right away? In this tutorial, I will show you how to build a model with the on-browser framework TensorFlow.js with data collected from your webcam and train on your browser. To make the model useful, we will turn a webcam into a controller for the legendary game - Pong. I have recorded a video as well.

It's impossible to read about the future of healthcare without encountering two pixelated vowels that, together, represent the hopes and fears of an industry seeking more intelligent solutions. Though the field of artificial intelligence (AI) has been around since 1956, it has made precious few contributions to medical practice. Only recently has the hype of machine-based learning begun to merge with reality. Confusion surrounding AI--its applications in healthcare and even its definition--remains widespread in popular media. Today, AI is shorthand for any task a computer can perform just as well as, if not better than, humans.

An Artificial General Intelligence (AGI) would be a machine capable of understanding the world as well as any human, and with the same capacity to learn how to carry out a huge range of tasks. AGI doesn't exist, but has featured in science-fiction stories for more than a century, and been popularized in modern times by films such as 2001: A Space Odyssey. Fictional depictions of AGI vary widely, although tend more towards the dystopian vision of intelligent machines eradicating or enslaving humanity, as seen in films like The Matrix or The Terminator. In such stories, AGI is often cast as either indifferent to human suffering or even bent on mankind's destruction. In contrast, utopian imaginings, such as Iain M Banks' Culture civilization novels, cast AGI as benevolent custodians, running egalitarian societies free of suffering, where inhabitants can pursue their passions and technology advances at a breathless pace. Whether these ideas would bear any resemblance to real-world AGI is unknowable since nothing of the sort has been created, or, according to many working in the field of AI, is even close to being created.

Yesterday we announced a new release of the Adversarial Robustness Toolbox, an open-source software library to support researchers and developers in defending neural networks against adversarial attacks. The new release provides a method for defending against poisoning and "backdoor" attacks in machine learning models. We announced the release at Black Hat USA, the world's leading information security event. Machine learning models are often trained on data from potentially untrustworthy sources, including crowd-sourced information, social media data, and user-generated data such as customer satisfaction ratings, purchasing history, or web traffic [1]. Recent work has shown that adversaries can introduce backdoors or "trojans" in machine learning models by poisoning training sets with malicious samples [2].

However even these reinforcement learning algorithms couldn't transfer what they'd learned about one task to acquiring a new task. In order to realize this achievement, DeepMind supercharged a reinforcement learning algorithm called A3C. In so-called actor-critic reinforcement learning, of which A3C is one variety, acting and learning are decoupled so that one neural network, the critic, evaluates the other, the actor. Together, they drive the learning process. This was already the state of the art, but DeepMind added a new off-policy correction algorithm called V-trace to the mix, which made the learning more efficient, and crucially, better able to achieve positive transfer between tasks.

Aug 20, 2018 (Heraldkeeper via COMTEX) -- A new research document is added in HTF MI database of 90 pages, titled as'Global Deep Learning Market (2018-2023)' with detailed analysis, Competitive landscape, forecast and strategies. The study covers geographic analysis that includes regions like North America, Europe, Asia-Pacific, Latin America, and the Middle East and Africa and important players/vendors such as Google, Microsoft, IBM, Intel etc. The report will help you gain market insights, future trends and growth prospects for forecast period of 2018-2023. In enterprise computing, deep learning is evolving into one of the most advanced technologies. Deep learning is a subset of machine learning in Artificial Intelligence (AI) that has networks capable of learning unsupervised, from data that is unstructured or unlabeled.