Deep Learning
Why good AI should be able to show its work
What's happening: Explainable AI, also sometimes called transparent AI, has become a top priority for nearly all the big companies in the AI field, including Microsoft, Google, Intel, IBM and Oracle. The topic is also expected to come up in Thursday's White House meeting on AI. That sounds straightforward, even obvious. But it actually isn't a feature built into many of the deep learning systems that are currently available. No one size fits all: AI was a huge topic at Google's I/O developer conference this week, with some focus on explainability as well.
Machine Learning for Integrating Data in Biology and Medicine: Principles, Practice, and Opportunities
Zitnik, Marinka, Nguyen, Francis, Wang, Bo, Leskovec, Jure, Goldenberg, Anna, Hoffman, Michael M.
New technologies have enabled the investigation of biology and human health at an unprecedented scale and in multiple dimensions. These dimensions include myriad properties describing genome, epigenome, transcriptome, microbiome, phenotype, and lifestyle. No single data type, however, can capture the complexity of all the factors relevant to understanding a phenomenon such as a disease. Integrative methods that combine data from multiple technologies have thus emerged as critical statistical and computational approaches. The key challenge in developing such approaches is the identification of effective models to provide a comprehensive and relevant systems view. An ideal method can answer a biological or medical question, identifying important features and predicting outcomes, by harnessing heterogeneous data across several dimensions of biological variation. In this Review, we describe the principles of data integration and discuss current methods and available implementations. We provide examples of successful data integration in biology and medicine. Finally, we discuss current challenges in biomedical integrative methods and our perspective on the future development of the field.
Accurate Uncertainties for Deep Learning Using Calibrated Regression
Kuleshov, Volodymyr, Fenner, Nathan, Ermon, Stefano
Methods for reasoning under uncertainty are a key building block of accurate and reliable machine learning systems. Bayesian methods provide a general framework to quantify uncertainty. However, because of model misspecification and the use of approximate inference, Bayesian uncertainty estimates are often inaccurate -- for example, a 90% credible interval may not contain the true outcome 90% of the time. Here, we propose a simple procedure for calibrating any regression algorithm; when applied to Bayesian and probabilistic models, it is guaranteed to produce calibrated uncertainty estimates given enough data. Our procedure is inspired by Platt scaling and extends previous work on classification. We evaluate this approach on Bayesian linear regression, feedforward, and recurrent neural networks, and find that it consistently outputs well-calibrated credible intervals while improving performance on time series forecasting and model-based reinforcement learning tasks.
Algorithms for solving optimization problems arising from deep neural net models: smooth problems
Kungurtsev, Vyacheslav, Pevny, Tomas
Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems. The resulting optimization problem to solve for the optimal vector minimizing the empirical risk is, however, highly nonlinear. This presents a challenge to application and development of appropriate optimization algorithms for solving the problem. In this paper, we summarize the primary challenges involved and present the case for a Newton-based method incorporating directions of negative curvature, including promising numerical results on data arising from security anomally deetection.
A New Benchmark and Progress Toward Improved Weakly Supervised Learning
In our work, we completely solve the previous Knowledge Matters problem using a generic model, pose a more difficult and scalable problem, All-Pairs, and advance this new problem by introducing a new learned, spatially-varying histogram model called TypeNet which outperforms conventional models on the problem. We present results on All-Pairs where our model achieves 100% test accuracy while the best ResNet models achieve 79% accuracy. In addition, our model is more than an order of magnitude smaller than Resnet-34. The challenge of solving larger-scale All-Pairs problems with high accuracy is presented to the community for investigation.
Algorithms for solving optimization problems arising from deep neural net models: nonsmooth problems
Kungurtsev, Vyacheslav, Pevny, Tomas
Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems. The resulting optimization problem to solve for the optimal vector minimizing the empirical risk is, however, highly nonconvex. This alone presents a challenge to application and development of appropriate optimization algorithms for solving the problem. However, in addition, there are a number of interesting problems for which the objective function is non- smooth and nonseparable. In this paper, we summarize the primary challenges involved, the state of the art, and present some numerical results on an interesting and representative class of problems.
Game-Theoretic Interpretability for Temporal Modeling
Lee, Guang-He, Alvarez-Melis, David, Jaakkola, Tommi S.
Interpretability has arisen as a key desideratum of machine learning models alongside performance. Approaches so far have been primarily concerned with fixed dimensional inputs emphasizing feature relevance or selection. In contrast, we focus on temporal modeling and the problem of tailoring the predictor, functionally, towards an interpretable family. To this end, we propose a co-operative game between the predictor and an explainer without any a priori restrictions on the functional class of the predictor. The goal of the explainer is to highlight, locally, how well the predictor conforms to the chosen interpretable family of temporal models. Our co-operative game is setup asymmetrically in terms of information sets for efficiency reasons. We develop and illustrate the framework in the context of temporal sequence models with examples.
8 Deep Learning Frameworks for Data Science Enthusiasts
AI coupled with the right deep learning framework has truly amplified the overall scale of what businesses can achieve and obtain within their domains. The machine learning paradigm is continuously evolving. The key is to shift towards developing machine learning models that run on mobile in order to make applications smarter and far more intelligent. Deep learning is what makes solving complex problems possible. As put in this article, Deep Learning is basically Machine Learning on steroids.
Unmasking A.I.'s Bias Problem
WHEN TAY MADE HER DEBUT in March 2016, Microsoft had high hopes for the artificial intelligence–powered "social chatbot." Like the automated, text-based chat programs that many people had already encountered on e-commerce sites and in customer service conversations, Tay could answer written questions; by doing so on Twitter and other social media, she could engage with the masses. But rather than simply doling out facts, Tay was engineered to converse in a more sophisticated way--one that had an emotional dimension. She would be able to show a sense of humor, to banter with people like a friend. Her creators had even engineered her to talk like a wisecracking teenage girl. When Twitter users asked Tay who her parents were, she might respond, "Oh a team of scientists in a Microsoft lab. They're what u would call my parents." If someone asked her how her day had been, she could quip, "omg totes exhausted."
F1 appoints Amazon as official Cloud and Machine Learning Provider - Pitpass.com
Amazon Web Services (AWS) has announced that the Formula One Group is moving the vast majority of its infrastructure from on-premises data centres to AWS, and standardizing on AWS's machine learning and data analytics services to accelerate its cloud transformation. Formula 1 will work with AWS to enhance its race strategies, data tracking systems, and digital broadcasts through a wide variety of AWS services - including Amazon SageMaker, a fully managed machine learning service that enables everyday developers and scientists to easily build and deploy machine learning models, AWS Lambda, AWS's pioneering event-driven serverless computing service, and AWS analytics services - to uncover never-before-seen metrics that will change the way fans and teams enjoy, experience, and participate in racing. Formula 1 has also selected AWS Elemental Media Services to power its video asset workflows, enhancing the viewing experience for its fans worldwide. Using Amazon SageMaker, Formula 1's data scientists are training deep learning models with more than 65 years of historical race data, stored in both Amazon DynamoDB and Amazon Glacier. With this information, Formula 1 can extract critical race performance statistics to make race predictions and give fans insight into the split-second decisions and strategies adopted by teams and drivers.