Theories of reinforcement learning in neuroscience have focused on two families of algorithms. Model-based algorithms achieve flexibility at computational expense, by rebuilding values from a model of the environment. We examine an intermediate class of algorithms, the successor representation (SR), which caches long-run state expectancies, blending model-free efficiency with model-based flexibility. Although previous reward revaluation studies distinguish model-free from model-based learning algorithms, such designs cannot discriminate between model-based and SR-based algorithms, both of which predict sensitivity to reward revaluation. However, changing the transition structure ('transition revaluation') should selectively impair revaluation for the SR.

Microsoft Cognitive Toolkit (CNTK) is a production-grade, open-source, deep-learning library. In the spirit of democratizing AI tools, CNTK embraces fully open development, is available on GitHub, and provides support for both Windows and Linux. These enhancements, combined with unparalleled scalability on NVIDIA hardware, were demonstrated by both NVIDIA at SuperComputing 2016 and Cray at NIPS 2016. These enhancements from the CNTK supported Microsoft in its recent breakthrough in speech recognition, reaching human parity in conversational speech. The toolkit is used in all kinds of deep learning, including image, video, speech, and text data.

Written by: Dr. John Kane, VP of Signal Processing & Data Science at Cogito There was something special about the prospect of having Interspeech 2017, the most prominent speech science and technology conference of the year, in Stockholm. Perhaps because of its rich history of speech science, including being home to great researchers like the late Gunnar Fant, who was responsible for pioneering work on the acoustics of speech production and speech synthesis. On arrival to the conference at Stockholm University, the specialness was evident as the opening ceremony was held in the architecturally impressive Aula Magna building where the FurHat robot delivered conference logistical information to the audience. There ensued a compelling four days of speech and language processing paper presentations that provided a window into the newest innovations in voice, emotion and speech technology. Check out the top themes from this year's interspeech conference below.

Neuroevolution is making a comeback. Prominent artificial intelligence labs and researchers are experimenting with it, a string of new successes have bolstered enthusiasm, and new opportunities for impact in deep learning are emerging. Maybe you haven't heard of neuroevolution in the midst of all the excitement over deep learning, but it's been lurking just below the surface, the subject of study for a small, enthusiastic research community for decades. And it's starting to gain more attention as people recognize its potential. Put simply, neuroevolution is a subfield within artificial intelligence (AI) and machine learning (ML) that consists of trying to trigger an evolutionary process similar to the one that produced our brains, except inside a computer. In other words, neuroevolution seeks to develop the means of evolving neural networks through evolutionary algorithms. When I first waded into AI research in the late 1990s, the idea that brains could be evolved inside computers resonated with my sense of adventure. At that time, it was an unusual, even obscure field, but I felt a deep curiosity and affinity. The result has been 20 years of my life thinking about this subject, and a slew of algorithms developed with outstanding colleagues over the years, such as NEAT, HyperNEAT, and novelty search. In this article, I hope to convey some of the excitement of neuroevolution as well as provide insight into its issues, but without the opaque technical jargon of scientific articles. I have also taken, in part, an autobiographical perspective, reflecting my own deep involvement within the field.

As of last month Alphabet Inc.'s AI division, Google DeepMind, has open-sourced their new machine learning platform DeepMind Lab. Artificial Intelligence is the technology of the moment, constantly debated and attracting massive attention from investors. Despite warnings from influential figures including Professor Stephen Hawking, Google's decision to open up their software to other developers is part of a mass movement to advance the capabilities of AI. Facebook open sourced its own deep learning software last year, and Elon Musk's non-profit organisation OpenAI recently released Universe, an open software platform that can be used to train AI systems. So, why have Google, OpenAI and others made these platforms public, and how will this affect the adoption of Artificial Intelligence and machine learning as a whole?

From time to time, we invite industry thought leaders to share their opinions and insights on current technology trends to the In The Making blog. The opinions in these blogs are their own, and do not necessarily reflect the views of IBM. Getting the right tool for the job is essential for anything from home improvement projects to launching satellites. I view the new trend of applying AI, deep learning and cognitive techniques to enterprise IT solutions as following that basic principle. Some tools are more complex and difficult to create than others, but they should all be viewed as a means to an end, not the end in itself.

Researchers have started developing artificial intelligence with imagination – AI that can reason through decisions and make plans for the future, without being bound by human instructions. Another way to put it would be imagining the consequences of actions before taking them, something we take for granted but which is much harder for robots to do. The team working at Google-owned lab DeepMind says this ability is going to be crucial in developing AI algorithms for the future, allowing systems to better adapt to changing conditions that they haven't been specifically programmed for. "When placing a glass on the edge of a table, for example, we will likely pause to consider how stable it is and whether it might fall," explain the researchers in a blog post. "On the basis of that imagined consequence we might readjust the glass to prevent it from falling and breaking." "If our algorithms are to develop equally sophisticated behaviours, they too must have the capability to'imagine' and reason about the future.

Summary: In the first part of this series we described the basics of Reinforcement Learning (RL). In this article we describe how deep learning is augmenting RL and a variety of challenges and considerations that need to be addressed in each implementation. In the first part of this series, Understanding Basic RL Models we described the basics of how reinforcement learning (RL) models are constructed and interpreted. RL systems can be constructed using policy gradient techniques which attempt to learn by directly mapping an observation to an action (the automated house look up table). Or they can be constructed using Q-Learning in which we train a neural net to calculate the estimated Q factor on the fly which is used when the state space gets large and complex.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

Conventional modeling approaches have found limitations in matching the increasingly detailed neural network structures and dynamics recorded in experiments to the diverse brain functionalities. On another approach, studies have demonstrated to train spiking neural networks for simple functions using supervised learning. Here, we introduce a modified SpikeProp learning algorithm, which achieved better learning stability in different activity states. In addition, we show biological realistic features such as lateral connections and sparse activities can be included in the network. We demonstrate the versatility of this framework by implementing three well-known temporal codes for different types of cognitive tasks, which are MNIST digits recognition, spatial coordinate transformation, and motor sequence generation. Moreover, we find several characteristic features have evolved alongside the task training, such as selective activity, excitatory-inhibitory balance, and weak pair-wise correlation. The coincidence between the self-evolved and experimentally observed features indicates their importance on the brain functionality. Our results suggest a unified setting in which diverse cognitive computations and mechanisms can be studied.