Deep Learning
Deep Learning Is Not Good Enough, We Need Bayesian Deep Learning for Safe AI
These results show that when we train on less data, or test on data which is significantly different from the training set, then our epistemic uncertainty increases drastically. However, our aleatoric uncertainty remains relatively constant, which it should because it is tested on the same problem with the same sensor. Next I'm going to discuss an interesting application of these ideas for multi-task learning. Multi-task learning aims to improve learning efficiency and prediction accuracy by learning multiple objectives from a shared representation. It is prevalent in many areas of machine learning, from NLP to speech recognition to computer vision.
What is Artificial Intelligence Machine Learning and Deep Learning
Artificial Intelligence (AI) has entered our daily lives like never before and we are yet to unravel the many other ways in which it could flourish. All tech giants such as Microsoft, Uber, Google, Facebook, Apple, Amazon, Oracle, Intel, IBM or Twitter are competing in the race to lead the market and acquire the most innovative and promising AI businesses. AI is already being used in everyday life with applications including speech recognition, smart cars, fraud detection, security surveillance, music recommendations and AI-powered personal virtual assistant such as Cortana (Microsoft), Siri (Apple) or Alexa (Amazon). Discussions on AI are generally dappled with the terms, 'Machine Learning' and'Deep Learning'. Moreover, they are often interchangeably used.
An AI Robot Learned How to Pick up Objects After Training Only in the Virtual World
While some researchers attempt to build artificial intelligences (AI) that can solve problems that humans might not have even thought of yet, others are focused on creating ones that do something most of us take for granted: pick things up. For a robot, knowing how to properly grasp and lift an object is no easy task. To address this issue, researchers at the University of California, Berkeley, trained a deep learning system on a cloud-based data set of more than a thousand objects, exposing it to each one's 3D shape and appearance, as well of the physics of grasping it. Afterward, they tested their system using physical objects that weren't included in its digital training set. When the system thought it had a better than 50 percent chance of successfully picking up a new object, it was actually able to do it 98 percent of the time -- all without having trained on any objects outside of the virtual world.
Using Deep Learning at Scale in Twitter's Timelines
For more than a year now since we enhanced our timeline to show the best Tweets for you first, we have worked to improve the underlying algorithms in order to surface content that is even more relevant to you. Today we are explaining how our ranking algorithm is powered by deep neural networks, leveraging the modeling capabilities and AI platform built by Cortex, one of our in-house AI teams at Twitter. In a nutshell: this means more relevant timelines now, and in the future, as this opens the door for us to use more of the many novelties that the deep learning community has to offer, especially in the areas of NLP (Natural Language Processing), conversation understanding, and media domains. Your timeline composition before the introduction of the ranking algorithm is easy to describe: all the Tweets from the people you follow since your last visit were gathered and shown in reverse-chronological order. Although the concept is simple to grasp, reliably serving this experience to the hundreds of millions of people on Twitter is an enormous infrastructural and operational challenge.
Unlocking Artificial Intelligence to beat Hunger
Artificial Intelligence (AI) technology is fast approaching the peak of the hype cycle for emerging tech, located where Bitcoin was a couple of years ago and 3D printers before that. Start-ups working on AI materialise on a daily basis, planning to disrupt and improve our businesses and lives. So, can this technology help improve the lives of the bottom billion? When most of us think of AI, our minds quickly turn to Terminator or Skynet or other scary sentient robots. Yet, we all interact with it on a daily basis, in ways most of us are only vaguely aware of.
San Jose becoming hub for artificial intelligence firms
Cheaper and older isn't typically associated with riches and success in the Bay Area tech scene. But that's just what San Jose is offering -- cheaper office rent and older tech workers -- to a rapidly expanding cohort of companies focused on artificial intelligence, the explosive new frontier in tech. "San Francisco has the gamers, we have the grownups," said San Jose Mayor Sam Liccardo. "We've got a very rich pool of talented, skilled workers." Artificial intelligence -- which can be broadly interpreted to include machine learning and the "deep learning" technology that resembles human thought -- is widely seen to be as revolutionary as the internet and mobile phones.
Auto-Encoding Sequential Monte Carlo
Le, Tuan Anh, Igl, Maximilian, Jin, Tom, Rainforth, Tom, Wood, Frank
Probabilistic machine learning [Ghahramani, 2015] allows us to model the structure and dependencies of latent variables and observational data as a joint probability distribution. Once a model is defined, we can perform inference to update our prior beliefs about latent variables in light of observed data to obtain the posterior distribution. The posterior can be used to answer any questions we might have about the latent quantities while coherently accounting for our uncertainty about the world. We introduce a method for simultaneous model learning and inference amortization [Gershman and Goodman, 2014], given an unlabeled dataset of observations. The model is specified partially, the rest being specified using a generative network whose weights are to be learned. Inference amortization refers to spending additional time before inference to obtain an amortization artifact which is used to speed up inference during test time.
Deep Learning for Patient-Specific Kidney Graft Survival Analysis
Luck, Margaux, Sylvain, Tristan, Cardinal, Hรฉloรฏse, Lodi, Andrea, Bengio, Yoshua
An accurate model of patient-specific kidney graft survival distributions can help to improve shared-decision making in the treatment and care of patients. In this paper, we propose a deep learning method that directly models the survival function instead of estimating the hazard function to predict survival times for graft patients based on the principle of multi-task learning. By learning to jointly predict the time of the event, and its rank in the cox partial log likelihood framework, our deep learning approach outperforms, in terms of survival time prediction quality and concordance index, other common methods for survival analysis, including the Cox Proportional Hazards model and a network trained on the cox partial log-likelihood.
Latent Intention Dialogue Models
Wen, Tsung-Hsien, Miao, Yishu, Blunsom, Phil, Young, Steve
Developing a dialogue agent that is capable of making autonomous decisions and communicating by natural language is one of the long-term goals of machine learning research. Traditional approaches either rely on hand-crafting a small state-action set for applying reinforcement learning that is not scalable or constructing deterministic models for learning dialogue sentences that fail to capture natural conversational variability. In this paper, we propose a Latent Intention Dialogue Model (LIDM) that employs a discrete latent variable to learn underlying dialogue intentions in the framework of neural variational inference. In a goal-oriented dialogue scenario, these latent intentions can be interpreted as actions guiding the generation of machine responses, which can be further refined autonomously by reinforcement learning. The experimental evaluation of LIDM shows that the model out-performs published benchmarks for both corpus-based and human evaluation, demonstrating the effectiveness of discrete latent variable models for learning goal-oriented dialogues.
Model Selection in Bayesian Neural Networks via Horseshoe Priors
Ghosh, Soumya, Doshi-Velez, Finale
Bayesian Neural Networks (BNNs) have recently received increasing attention for their ability to provide well-calibrated posterior uncertainties. However, model selection---even choosing the number of nodes---remains an open question. In this work, we apply a horseshoe prior over node pre-activations of a Bayesian neural network, which effectively turns off nodes that do not help explain the data. We demonstrate that our prior prevents the BNN from under-fitting even when the number of nodes required is grossly over-estimated. Moreover, this model selection over the number of nodes doesn't come at the expense of predictive or computational performance; in fact, we learn smaller networks with comparable predictive performance to current approaches.