Deep Learning
France Initiatives to Tackle the Challenges of Artificial Intelligence
Five'table ronde' or round table were organised mostly with academics on the different aspects of the societal moves due to Artificial Intelligence (AI or IA in French): It was pointed that some milestone progress on deep learning has been achieved. Machines have surpassed human champions in most intellectually challenging games, including Chess, Scrabble, Othello, even Jeopardy. On March 2016, the Google AlphaGo DeepMind's Artificial Intelligence program beat Lee Sedol, the strongest Go player in the world. Go--a 2,500-year-old game is far more complex than Chess. An exceptional powerful computer had to process more than 30 million moves.
Global Bigdata Conference
Firstly, my response contains some bias, because I work at Google Brain and I really like it there. My opinions are my own, and I do not speak for the rest of my colleagues or Alphabet as a whole. I rank "leaders in AI research" among tech companies as follows: I would say Deepmind is probably #1 right now, in terms of AI research. Their publications are highly respected within the research community, and span a myriad of topics such as Deep Reinforcement Learning, Bayesian Neural Nets, Robotics, transfer learning, and others. Being London-based, they recruit heavily from Oxford and Cambridge, which are great ML feeder programs in Europe.
What Does Artificial Intelligence See In A Quarter Billion Global News Photographs?
What would it look like to ask a deep learning AI system to watch every political television advertisement of the 2016 presidential campaign season for two months and describe what it sees? That was the question I asked last February when I collaborated with the Internet Archive to take all 267 political ads they had identified (which had aired a collective 72,807 times as monitored by the Archive) and ran them frame-by-frame through Google's Cloud Vision API, producing what is likely the first large-scale application of production deep learning algorithms to describe the visual narratives of political advertising on television. Now, what if we took this same approach and instead of examining television, we looked at a quarter billion news photographs compiled from online news outlets in nearly every country of the world over the course of 2016? What would AI see in that vast archive of the visual narratives of the world's media? Google's Cloud Vision API is a commercial cloud service that accepts as input any arbitrary photograph and uses deep learning algorithms to catalog a wealth of data about each image, including a list of objects and activities it depicts, recognizable logos, OCR text recognition in almost 80 languages, levels of violence, an estimate of visual sentiment and even the precise location on earth the image appears to depict.
Ternary Neural Networks for Resource-Efficient AI Applications
Alemdar, Hande, Leroy, Vincent, Prost-Boucle, Adrien, Pรฉtrot, Frรฉdรฉric
The computation and storage requirements for Deep Neural Networks (DNNs) are usually high. This issue limits their deployability on ubiquitous computing devices such as smart phones, wearables and autonomous drones. In this paper, we propose ternary neural networks (TNNs) in order to make deep learning more resource-efficient. We train these TNNs using a teacher-student approach based on a novel, layer-wise greedy methodology. Thanks to our two-stage training procedure, the teacher network is still able to use state-of-the-art methods such as dropout and batch normalization to increase accuracy and reduce training time. Using only ternary weights and activations, the student ternary network learns to mimic the behavior of its teacher network without using any multiplication. Unlike its -1,1 binary counterparts, a ternary neural network inherently prunes the smaller weights by setting them to zero during training. This makes them sparser and thus more energy-efficient. We design a purpose-built hardware architecture for TNNs and implement it on FPGA and ASIC. We evaluate TNNs on several benchmark datasets and demonstrate up to 3.1x better energy efficiency with respect to the state of the art while also improving accuracy.
TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
Dieng, Adji B., Wang, Chong, Gao, Jianfeng, Paisley, John
In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based language model designed to directly capture the global semantic meaning relating words in a document via latent topics. Because of their sequential nature, RNNs are good at capturing the local structure of a word sequence - both semantic and syntactic - but might face difficulty remembering long-range dependencies. Intuitively, these long-range dependencies are of semantic nature. In contrast, latent topic models are able to capture the global underlying semantic structure of a document but do not account for word ordering. The proposed TopicRNN model integrates the merits of RNNs and latent topic models: it captures local (syntactic) dependencies using an RNN and global (semantic) dependencies using latent topics. Unlike previous work on contextual RNN language modeling, our model is learned end-to-end. Empirical results on word prediction show that TopicRNN outperforms existing contextual RNN baselines. In addition, TopicRNN can be used as an unsupervised feature extractor for documents. We do this for sentiment analysis on the IMDB movie review dataset and report an error rate of $6.28\%$. This is comparable to the state-of-the-art $5.91\%$ resulting from a semi-supervised approach. Finally, TopicRNN also yields sensible topics, making it a useful alternative to document models such as latent Dirichlet allocation.
RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
Choi, Edward, Bahadori, Mohammad Taha, Kulas, Joshua A., Schuetz, Andy, Stewart, Walter F., Sun, Jimeng
Accuracy and interpretability are two dominant features of successful predictive models. Typically, a choice must be made in favor of complex black box models such as recurrent neural networks (RNN) for accuracy versus less accurate but more interpretable traditional models such as logistic regression. This tradeoff poses challenges in medicine where both accuracy and interpretability are important. We addressed this challenge by developing the REverse Time AttentIoN model (RETAIN) for application to Electronic Health Records (EHR) data. RETAIN achieves high accuracy while remaining clinically interpretable and is based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits (e.g. key diagnoses). RETAIN mimics physician practice by attending the EHR data in a reverse time order so that recent clinical visits are likely to receive higher attention. RETAIN was tested on a large health system EHR dataset with 14 million visits completed by 263K patients over an 8 year period and demonstrated predictive accuracy and computational scalability comparable to state-of-the-art methods such as RNN, and ease of interpretability comparable to traditional models.
How Zendesk Serves TensorFlow Models in Production โ Zendesk Engineering
At Zendesk we are developing a series of machine learning products, the most recent of which is Automatic Answers. It uses machine learning to interpret user questions and responds with relevant knowledge base articles. When a customer has a question, complaint or enquiry, they may submit their request online. Once their request is received, Automatic Answers will analyse the request and suggest relevant articles which may best assist with the customer's request via email. Automatic answers uses a class of state-of-the-art machine learning algorithms known as deep learning to identify relevant articles.
Achieving the World's Fastest Training Speed! The Latest Deep Learning Technologies to Create a High-Accuracy AI : FUJITSU JOURNAL
Our exposure to the term deep learning has recently increased. It refers to a technology using a neural network * that is repetitively trained on large data sets. It is also a method of improving recognition and categorization accuracy. Recently, research on deep learning is advancing rapidly, and the technology has achieved higher image, character, and voice recognition accuracy than humans. In order to improve the accuracy of these processes, the use of large data sets is required in deep learning.
How Artificial Intelligence Will Change Medical Imaging
Artificial intelligence (AI) has captured the imagination and attention of doctors over the past couple years as several companies and large research hospitals work to perfect these systems for clinical use. The first concrete examples of how AI (also called deep learning, machine learning or artificial neural networks) will help clinicians are now being commercialized. These systems may offer a paradigm shift in how clinicians work in an effort to significantly boost workflow efficiency, while at the same time improving care and patient throughput. Today, one of the biggest problems facing physicians and clinicians in general is the overload of too much patient information to sift through. This rapid accumulation of electronic data is thanks to the advent of electronic medical records (EMRs) and the capture of all sorts of data about a patient that was not previously recorded, or at least not easily data mined.
Character-level Convolutional Networks for Text Classification
One of the common natural language understanding problems is text classification. Over last few decades, machine learning researchers have been moving from the simplest "bag of words" model to more sophisticated models for text classification. Bag of words model uses only information about which words are used in the text. Adding TFIDF to the bag of words helps to track relevancy of each word to the document. Bag of n-grams enables using partial information about structure of the text. Recurrent neural networks, like LSTM, can capture dependencies between words even if they are far from each other.