Neural approaches to sequence labeling often use a Conditional Random Field (CRF) to model their output dependencies, while Recurrent Neural Networks (RNN) are used for the same purpose in other tasks. We set out to establish RNNs as an attractive alternative to CRFs for sequence labeling. To do so, we address one of the RNN's most prominent shortcomings, the fact that it is not exposed to its own errors with the maximum-likelihood training. We frame the prediction of the output sequence as a sequential decision-making process, where we train the network with an adjusted actor-critic algorithm (AC-RNN). We comprehensively compare this strategy with maximum-likelihood training for both RNNs and CRFs on three structured-output tasks. The proposed AC-RNN efficiently matches the performance of the CRF on NER and CCG tagging, and outperforms it on Machine Transliteration. We also show that our training strategy is significantly better than other techniques for addressing RNN's exposure bias, such as Scheduled Sampling, and Self-Critical policy training.

We propose Neural Entity Reasoner (NE-Reasoner), a framework to introduce global consistency of recognized entities into Neural Reasoner over Named Entity Recognition (NER) task. Given an input sentence, the NE-Reasoner layer can infer over multiple entities to increase the global consistency of output labels, which then be transfered into entities for the input of next layer. NE-Reasoner inherits and develops some features from Neural Reasoner 1) a symbolic memory, allowing it to exchange entities between layers. 2) the specific interaction-pooling mechanism, allowing it to connect each local word to multiple global entities, and 3) the deep architecture, allowing it to bootstrap the recognized entity set from coarse to fine. Like human beings, NE-Reasoner is able to accommodate ambiguous words and Name Entities that rarely or never met before. Despite the symbolic information the model introduced, NE-Reasoner can still be trained effectively in an end-to-end manner via parameter sharing strategy. NE-Reasoner can outperform conventional NER models in most cases on both English and Chinese NER datasets. For example, it achieves state-of-art on CoNLL-2003 English NER dataset.

Responsibilities: Develop, test, and improve models and algorithms to sort and automatically analyze datasets and extract organizational insight.

R.P. and M.A.D. designed the study, analyzed and interpreted results and wrote the paper. This work was internally funded by Verily Life Sciences and Google Inc.

The next article will be about Sequence modeling with Neural Networks. We will learn how to model sequences with a focus on Recurrent Neural Networks (RNNs) and their short-term memory and Long Short Term Memory (LSTM) and their ability to keep track of information throughout many timesteps. Bio: Zied Haj-Yahia is Senior Data Scientist at Capgemini Consulting. He specializes in building predictive models utilizing both traditional statistical methods and modern machine learning techniques. He also runs some workshops for university students (ESSEC, HEC, Ecole polytechnique) interested in Data Science and its applications. He is the co-founder of Global International Trading (GIT), a central purchasing office based in Paris.

On this week's Making Data Simple Podcast, we are joined by Steve Moore, IBM senior content designer and story strategist, along with Aishwarya Srinivasan, IBM data scientist and deep learning researcher. Aishwarya discusses what it means to be a unicorn in an industry and why it is useful to pair textbook learning from schooling with hands-on industry experience. Aishwarya, Steve, and Al explore how to use reinforcement learning to make better data decisions.

AI, data and cloud are ushering the next wave of transformative innovations across industries. With Azure AI, our goal is to empower organizations to apply AI across the spectrum of their business to engage customers, empower employees, optimize operations and transform products. We see customers using Azure AI to derive tangible benefits across three key solution areas. Today, at Microsoft Ignite, we are excited to announce a range of innovations across these areas to make Azure the best place for AI. Let me walk you through them.

This development has many exciting prospects. One of them is that all the work AI researchers have put into musical applications of neural nets is suddenly available to use in web apps. We can now use generative musical deep learning models and build interactive web-based experiences on top of them, with the help of Web Audio and the rest of the web platform. In this talk I will present some recent experiments I've made using deep neural nets and Web Audio in the browser. We'll see how recurrent neural networks can be used to build melodic "autocompletion" tools and arpeggiators, as well as generative drum patterns.

Deep learning continues to be the hottest thing in data science. Deep learning frameworks are changing rapidly. Just five years ago, none of the leaders other than Theano were even around. I wanted to find evidence for which frameworks merit attention, so I developed this power ranking. I used 11 data sources across 7 distinct categories to gauge framework usage, interest, and popularity. Then I weighted and combined the data in this Kaggle Kernel. UPDATE SEPT 20, 2018: Due to popular demand, I expanded the frameworks evaluated to include Caffe, Deeplearning4J, Caffe2, and Chainer. Now all deep learning frameworks with more than 1% reported usage on KDNuggets usage survey are included.

From voice translation to self-driving automobile, AI's impact in everyday life will become more and more apparent this year. The AI and deep learning market will experience even more rapid technological advancement, very rapid growth and adoption, and increasing competition for both hardware and software platforms. While AI fears will remain, the public will become more cognisant and comfortable with social media AI applications. Deep learning training lends itself to what we call "High Density Processing". High density processing applies when algorithms are computationally intensive, having higher ratios of compute operations per byte of memory bandwidth.