This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows significant improvement over a strong RNN-based encoder-decoder baseline and achieved its best results on an abstractive summarization benchmark.

Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointing decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.

Keep those hands on the keyboard! We can do a lot on OSX and Linux without touching a mouse or GUI. Awesome command line tools for *N*X derivatives have been around since day one, and have expanded to include Python, Go, NodeJS, and hybrid tools. Even if you are not only running your pipeline through the command line, you can call most of these tools from Apache NiFi for processing. The book Data Science at the Command Line and GitHub offer an amazing set of quality tools to do a lot of pre- and post-processing and allow for a lot of transformations. I highly recommend looking at all of these amazing tools.

Summary: Here's some background on how 3rd generation Spiking Neural Nets are progressing and news about a first commercial rollout. Recently we wrote about the development of AI and neural nets beyond the second generation Convolutional and Recurrent Neural Nets (CNNs / RNNs) which have come on so strong and dominate the current conversation about deep learning. Our research shows that the next generation of neural nets is most likely to be led by Spiking Neural Nets (SNNs) that are a return to the'strong' AI tradition and closely mimic actual brain function. Unlike CNNs that fire signals to every one of their deep layer connections every time, SNNs are modeled after the fact that in the brain neurons do not constantly communicate with one another. Rather they communicate in spikes of signals or more correctly short trains of spiking signals.

Artificial intelligence is learning to take on an increasing number of sophisticated tasks. Google Deepmind's AI is now able to imitate human speech, and just this past August IBM's Watson successfully diagnosed a rare case of leukemia. Rather than viewing these advances as threats to job security, we can look at them as opportunities for AI to fill in critical gaps in existing service providers, such as mental healthcare professionals. In the US alone, nearly eight percent of the population suffers from depression (that's about one in every 13 American adults), and yet about 45 percent of this population does not seek professional care due to the costs. There are many barriers to getting quality mental healthcare, from searching for a provider who's within your insurance network to screening multiple potential therapists in order to find someone you feel comfortable speaking with.

Deep learning is a recent trend in machine learning that models highly non-linear representations of data. In the past years, deep learning has gained a tremendous momentum and prevalence for a variety of applications (Wikipedia 2016a). Among these are image and speech recognition, driverless cars, natural language processing and many more. Interestingly, the majority of mathematical concepts for deep learning have been known for decades. However, it is only through several recent developments that the full potential of deep learning has been unleashed (Nair and Hinton 2010; Srivastava et al. 2014). Previously, it was hard to train artificial neural networks due to vanishing gradients and overfitting problems.

Perhaps the most positive technical theme of 2016 was the long-delayed triumph of artificial intelligence, machine learning, and in particular deep learning. In this article we'll discuss what that means and how you might make use of deep learning yourself. Perhaps you noticed in the fall of 2016 that Google Translate suddenly went from producing, on the average, word salad with a vague connection to the original language to emitting polished, coherent sentences more often than not -- at least for supported language pairs, such as English-French, English-Chinese, and English-Japanese. That dramatic improvement was the result of a nine-month concerted effort by the Google Brain and Google Translate teams to revamp Translate from using its old phrase-based statistical machine translation algorithms to working with a neural network trained with deep learning and word embeddings employing Google's TensorFlow framework. The researchers working on the conversion had access to a huge corpus of translations from which to train their networks, but they soon discovered that they needed thousands of GPUs for training and would have to create a new kind of chip, a Tensor Processing Unit (TPU), to run Translate on their trained neural networks at scale.

When our robot overlords arrive, will they decide to kill us or cooperate with us? New research from DeepMind, Alphabet Inc.'s London-based artificial intelligence unit, could ultimately shed light on this fundamental question. They have been investigating the conditions in which reward-optimizing beings, whether human or robot, would choose to cooperate, rather than compete. The answer could have implications for how computer intelligence may eventually be deployed to manage complex systems such as an economy, city traffic flows, or environmental policy. Joel Leibo, the lead author of a paper DeepMind published online Thursday, said in an e-mail that his team's research indicates that whether agents learn to cooperate or compete depends strongly on the environment in which they operate.

I believe that there exist classic deep learning papers which are worth reading regardless of their application areas. Rather than providing overwhelming amount of papers, I would like to provide a curated list of the classic deep learning papers which can be considered as must-reads in some research areas. Please read the contributing guide before you make a pull request. Distinguished deep learning researchers who have published 3 ( 6) papers on the awesome list (The papers in Hardware / Software, Papers Worth Reading, Classic Papers sections are excluded in counting.) Thank you for all your contributions.

Graphics chipmaker Nvidia easily topped second quarter earnings targets Thursday after the bell. The company posted record-revenue for the quarter, and once again credits strong sales of its GPUs and deep learning technology for the boost on its balance sheet. See how the cloud is disrupting traditional operating models for IT departments and entire organizations. Nvidia co-founder and CEO Jen-Hsun Huang said the convergence of graphics, computer vision and artificial intelligence is fueling growth across the company's specialized platforms, including gaming, pro visualization, datacenter and automotive. "We are more excited than ever about the impact of deep learning and AI, which will touch every industry and market. We have made significant investments over the past five years to evolve our entire GPU computing stack for deep learning. Now, we are well positioned to partner with researchers and developers all over the world to democratize this powerful technology and invent its future," Huang said.