Deep Learning
How Deep Learning Is Transforming Brain Mapping
Thanks to deep learning, the tricky business of making brain atlases just got a lot easier. Brain maps are all the rage these days. From rainbow-colored dots that highlight neurons or gene expression across the brain, to neon "brush strokes" that represent neural connections, every few months seem to welcome a new brain map. Without doubt, these maps are invaluable for connecting the macro (the brain's architecture) to the micro (genetic profiles, protein expression, neural networks) across space and time. Scientists can now compare brain images from their own experiments to a standard resource.
Deep learning for detecting inappropriate content in text
Today, there are a large number of online discussion fora on the internet which are meant for users to express, discuss and exchange their views and opinions on various topics. In such fora, it has been often observed that user conversations sometimes quickly derail and become inappropriate such as hurling abuses, passing rude and discourteous comments on individuals or certain groups/communities. Similarly, some virtual agents or bots have also been found to respond back to users with inappropriate messages. As a result, inappropriate messages or comments are turning into an online menace slowly degrading the effectiveness of user experiences. Hence, automatic detection and filtering of such inappropriate language has become an important problem for improving the quality of conversations with users as well as virtual agents.
Building a modern data and analytics architecture
We expect the landscape to be an integrated edge-to-core-to-cloud solution enabling what today is called IoT, Big Data, Fast Data and AI. Each time a promising new technology emerges, we seem to go through a period where it is proposed to be the solution to everything--until we reconcile how that technology fits into the bigger picture. Such is the case with artificial intelligence (AI). Clearly the advancements in deep learning will create new classes of solutions but rather than being a standalone solution, we are just now beginning to see how it fits into our IT landscape. AI emerges at a time when several other shifts in analytics technology are occurring.
How Data Center Infrastructure Management (DCIM) Response to AI and Machine Learning
We have an almost mystical faith in the ability of artificial intelligence (AI) to understand and solve problems. It's being applied across many areas of our daily lives and, as a result, the hardware to enable this is starting to populate our data centers. Data centers in themselves present an array of complex problems, including optimization and prediction. So, how about using this miracle technology to improve our facilities? Machine learning, and especially deep learning, can examine a large set of data, and find patterns within it that do not depend on the model that humans would use to understand and predict that data.
Deep Learning with Merrill Grambell Improv Asylum
We live in a world where everything is connected, smart fridges, salt dispensers, egg timers, and even hair brushes. That sounds like a Ummm, ok-ish idea. Well, that's what this show is. Deep Learning with Merrill Grambell is the first show completely hosted by artificial intelligence as it interacts with real-life comedians, musicians, technologists and other interesting guests from all over the world. It's sort of as if Clippy from Microsoft Word and Bonzi Buddy got together and made a talk show.
Uncertainty-aware Model-based Policy Optimization
Vuong, Tung-Long, Tran, Kenneth
Model-based reinforcement learning has the potential to be more sample efficient than model-free approaches. However, existing model-based methods are vulnerable to model bias, which leads to poor generalization and asymptotic performance compared to model-free counterparts. In addition, they are typically based on the model predictive control (MPC) framework, which not only is computationally inefficient at decision time but also does not enable policy transfer due to the lack of an explicit policy representation. In this paper, we propose a novel uncertainty-aware model-based policy optimization framework which solves those issues. In this framework, the agent simultaneously learns an uncertainty-aware dynamics model and optimizes the policy according to these learned models. In the optimization step, the policy gradient is computed by automatic differentiation through the models. With respect to sample efficiency alone, our approach shows promising results on challenging continuous control benchmarks with competitive asymptotic performance and significantly lower sample complexity than state-of-the-art baselines.
Deep Conversational Recommender in Travel
Liao, Lizi, Takanobu, Ryuichi, Ma, Yunshan, Yang, Xun, Huang, Minlie, Chua, Tat-Seng
When traveling to a foreign country, we are often in dire need of an intelligent conversational agent to provide instant and informative responses to our various queries. However, to build such a travel agent is non-trivial. First of all, travel naturally involves several sub-tasks such as hotel reservation, restaurant recommendation and taxi booking etc, which invokes the need for global topic control. Secondly, the agent should consider various constraints like price or distance given by the user to recommend an appropriate venue. In this paper, we present a Deep Conversational Recommender (DCR) and apply to travel. It augments the sequence-to-sequence (seq2seq) models with a neural latent topic component to better guide response generation and make the training easier. To consider the various constraints for venue recommendation, we leverage a graph convolutional network (GCN) based approach to capture the relationships between different venues and the match between venue and dialog context. For response generation, we combine the topic-based component with the idea of pointer networks, which allows us to effectively incorporate recommendation results. We perform extensive evaluation on a multi-turn task-oriented dialog dataset in travel domain and the results show that our method achieves superior performance as compared to a wide range of baselines.
Importance Estimation for Neural Network Pruning
Molchanov, Pavlo, Mallya, Arun, Tyree, Stephen, Frosio, Iuri, Kautz, Jan
Structural pruning of neural network parameters reduces computation, energy, and memory transfer costs during inference. We propose a novel method that estimates the contribution of a neuron (filter) to the final loss and iteratively removes those with smaller scores. We describe two variations of our method using the first and second-order Taylor expansions to approximate a filter's contribution. Both methods scale consistently across any network layer without requiring per-layer sensitivity analysis and can be applied to any kind of layer, including skip connections. For modern networks trained on ImageNet, we measured experimentally a high (>93%) correlation between the contribution computed by our methods and a reliable estimate of the true importance. Pruning with the proposed methods leads to an improvement over state-of-the-art in terms of accuracy, FLOPs, and parameter reduction. On ResNet-101, we achieve a 40% FLOPS reduction by removing 30% of the parameters, with a loss of 0.02% in the top-1 accuracy on ImageNet. Code is available at https://github.com/NVlabs/Taylor_pruning.
Explaining Deep Learning Models with Constrained Adversarial Examples
Moore, Jonathan, Hammerla, Nils, Watkins, Chris
Machine learning algorithms generally suffer from a problem of explainability. Given a classification result from a model, it is typically hard to determine what caused the decision to be made, and to give an informative explanation. We explore a new method of generating counterfactual explanations, which instead of explaining why a particular classification was made explain how a different outcome can be achieved. This gives the recipients of the explanation a better way to understand the outcome, and provides an actionable suggestion. We show that the introduced method of Constrained Adversarial Examples (CADEX) can be used in real world applications, and yields explanations which incorporate business or domain constraints such as handling categorical attributes and range constraints.
Interpretable Image Recognition with Hierarchical Prototypes
Hase, Peter, Chen, Chaofan, Li, Oscar, Rudin, Cynthia
Vision models are interpretable when they classify objects on the basis of features that a person can directly understand. Recently, methods relying on visual feature prototypes have been developed for this purpose. However, in contrast to how humans categorize objects, these approaches have not yet made use of any taxonomical organization of class labels. With such an approach, for instance, we may see why a chimpanzee is classified as a chimpanzee, but not why it was considered to be a primate or even an animal. In this work we introduce a model that uses hierarchically organized prototypes to classify objects at every level in a predefined taxonomy. Hence, we may find distinct explanations for the prediction an image receives at each level of the taxonomy. The hierarchical prototypes enable the model to perform another important task: interpretably classifying images from previously unseen classes at the level of the taxonomy to which they correctly relate, e.g. classifying a hand gun as a weapon, when the only weapons in the training data are rifles. With a subset of ImageNet, we test our model against its counterpart black-box model on two tasks: 1) classification of data from familiar classes, and 2) classification of data from previously unseen classes at the appropriate level in the taxonomy. We find that our model performs approximately as well as its counterpart black-box model while allowing for each classification to be interpreted.