Deep Learning
Using Recurrent Neural Networks to Predict Player Performance
The future might not be far away. Editor's Note: This article initially was a presentation at the marvelous 2017 Saberseminar. Advancements in technology allow Buck Showalter to unplug a USB drive from a port behind his left ear and transfer it to an iPad. They don't call them iPads anymore, but there's no need to be bombarded with a bunch of unnecessary jargon right now. If that's what you came here for, sit tight.
Minds and Machines
A few months ago, Andy McAfee and Erik Brynjolfsson published Machine, Platform, Crowd: Harnessing Our Digital Future, - their third book on the impact of the 21st century digital revolution on the economy and society, - following the publication of The Second Machine Age in 2014 and Race Against the Machine in 2011. Brynjolfsson and McAfee are professor and research scientist respectively at MIT's Sloan School of Management, as well as co-Directors of MIT's Initiative on the Digital Economy. The book is organized into three sections, each focused on a major trend that's reshaping the business world: the rapidly expanding capabilities of machines; the emergence of large, asset-light platform companies; and the ability to now leverage the knowledge, expertise and enthusiasm of the crowd. These three trends are combining into a triple revolution, causing companies to rethink the balance between minds and machines; between products and platforms; and between the core and the crowd. I cannot possible do justice to all three trends in one blog, so let me summarize the key themes of the Mind and Machine section, which I found to be an excellent explanation of the current state of AI.
Artificial Intelligence and the Military
The Department of Defense (DoD) is increasingly interested in Artificial Intelligence (AI). During a recent trip to Amazon, Google, and other Silicon Valley companies, Secretary of Defense James Mattis remarked that AI has "got to be better integrated by the DoD." What do we mean by the term AI? In particular, what does "deep learning" mean? What are the advantages, disadvantages, and risks of using AI?
AI won't go anywhere unless it has empathy
Rapid advances in artificial intelligence have improved the human factor of machines. Like humans, machines can be influenced by people with varying ethical values. In looking at AI's brief history, exemplified through innovations like chatbots, we encounter ethical challenges that we must overcome to ensure AI is not adversely molded, to the detriment of users. Some earlier iterations of AI were marketed as smart search capabilities. Machines learned from their users and made personalized recommendations based on the preferences and behavior of both individuals and profiled segments.
MACHINE LEARNING And DEEP LEARNING For Beginners
Are you interested in the cutting edge of artificial intelligence? Do you want to understand how your phone can understand your voice? Or do you perhaps want to learn what happens when statistics, biology, and psychology combine with computer science? Maybe you just want to know how a robot can actually learn to walk? If these questions are on your mind then the answer you are looking for is machine learning.
Amazon Web Services, Inc.
AWS offers a family of intelligent services that provide cloud-native machine learning and deep learning technologies to address your different use cases and needs. For developers looking to add managed AI services to their applications, AWS brings natural language understanding (NLU) and automatic speech recognition (ASR) with Amazon Lex, visual search and image recognition with Amazon Rekognition, text-to-speech (TTS) with Amazon Polly, and developer-focused machine learning with Amazon Machine Learning. For more in-depth deep learning applications, the AWS Deep Learning AMI lets you run deep learning in the cloud, at any scale. Launch instances of the AMI, pre-installed with open source deep learning engines (Apache MXNet, TensorFlow, Caffe, Theano, Torch and Keras), to train sophisticated, custom AI models, experiment with new algorithms, and learn new deep learning skills and techniques; all backed by auto-scaling clusters of GPU-based instances. Whether you're just getting started with AI or you're a deep learning expert, this session will provide a meaningful overview of how to improve scale and efficiency with the AWS Cloud.
Deep Learning: A Practitioner's Approach: 9781491914250: Computer Science Books @ Amazon.com
I am hoping this book earns five stars, and will come back and update this rating if the typo I found on day one is an aberration. I ordered this book back in May and was very pleased to get it; so far it is excellent and exactly at the level I need: I am a sometime practitioner of machine learning and AI using a range of open source and off-the-shelf tools. I have moved more strongly into Python as i mostly deal with text and NLTK and python have bee easier / faster for me to pick up and use than java-syntric approaches. So moving into deep learning is a big step but I feel I am well prepared, and the level and degree of "refreshers" here, from linear algebra tp statistics are hitting just the right depth and tone. I initially thought my Kindle software was broken when searching for the first occurrence of "SGD" didn't show up; I remembered it was referred to as the "canonical" solution to solving a system of linear in an iterative fashion, but forgot what it stood for.
Verifying Properties of Binarized Deep Neural Networks
Narodytska, Nina, Kasiviswanathan, Shiva Prasad, Ryzhyk, Leonid, Sagiv, Mooly, Walsh, Toby
Understanding properties of deep neural networks is an important challenge in deep learning. In this paper, we take a step in this direction by proposing a rigorous way of verifying properties of a popular class of neural networks, Binarized Neural Networks, using the well-developed means of Boolean satisfiability. Our main contribution is a construction that creates a representation of a binarized neural network as a Boolean formula. Our encoding is the first exact Boolean representation of a deep neural network. Using this encoding, we leverage the power of modern SAT solvers along with a proposed counterexample-guided search procedure to verify various properties of these networks. A particular focus will be on the critical property of robustness to adversarial perturbations. For this property, our experimental results demonstrate that our approach scales to medium-size deep neural networks used in image classification tasks. To the best of our knowledge, this is the first work on verifying properties of deep neural networks using an exact Boolean encoding of the network.
Interactive Music Generation with Positional Constraints using Anticipation-RNNs
Hadjeres, Gaëtan, Nielsen, Frank
Recurrent Neural Networks (RNNS) are now widely used on sequence generation tasks due to their ability to learn long-range dependencies and to generate sequences of arbitrary length. However, their left-to-right generation procedure only allows a limited control from a potential user which makes them unsuitable for interactive and creative usages such as interactive music generation. This paper introduces a novel architecture called Anticipation-RNN which possesses the assets of the RNN-based generative models while allowing to enforce user-defined positional constraints. We demonstrate its efficiency on the task of generating melodies satisfying positional constraints in the style of the soprano parts of the J.S. Bach chorale harmonizations. Sampling using the Anticipation-RNN is of the same order of complexity than sampling from the traditional RNN model. This fast and interactive generation of musical sequences opens ways to devise real-time systems that could be used for creative purposes.
Deep Automated Multi-task Learning
Multi-task learning (MTL) has recently contributed to learning better representations in service of various NLP tasks. MTL aims at improving the performance of a primary task, by jointly training on a secondary task. This paper introduces automated tasks, which exploit the sequential nature of the input data, as secondary tasks in an MTL model. We explore next word prediction, next character prediction, and missing word completion as potential automated tasks. Our results show that training on a primary task in parallel with a secondary automated task improves both the convergence speed and accuracy for the primary task. We suggest two methods for augmenting an existing network with automated tasks and establish better performance in topic prediction, sentiment analysis, and hashtag recommendation. Finally, we show that the MTL models can perform well on datasets that are small and colloquial by nature.