Animal behavior is driven by multiple brain regions working in parallel with distinct control policies. We present a biologically plausible model of off-policy reinforcement learning in the basal ganglia, which enables learning in such an architecture. The model accounts for action-related modulation of dopamine activity that is not captured by previous models that implement on-policy algorithms. In particular, the model predicts that dopamine activity signals a combination of reward prediction error (as in classic models) and "action surprise," a measure of how unexpected an action is relative to the basal ganglia's current policy. In the presence of the action surprise term, the model implements an approximate form of Q-learning.

Neural Information Processing Systems http://nips.cc/

Finally, we validated our method using a previously published dataset.

Relationship columnists and pop psychologists have long claimed that men and women are wired differently, and a new study has proven them correct. Scientists developed an artificial intelligence model that was able to tell the difference between scans of men's and women's brain activity with more than 90-percent accuracy. Most of these differences are in the default mode network, striatum, and limbic network - areas involved in a wide range of processes including daydreaming, remembering the past, planning for the future, making decisions, and smelling. With these results, scientists at Stanford Medicine add a new piece to the puzzle, supporting the idea that biological sex shapes the brain. The researchers said they are optimistic that this work will help shed light on brain conditions that affect men and women differently.

Recent experimental studies have focused on the specialization of different neural structures for different types of instrumental behavior. Recent theoretical work has provided normative accounts for why there should be more than one control system, and how the output of different controllers can be integrated. Two par- ticlar controllers have been identified, one associated with a forward model and the prefrontal cortex and a second associated with computationally simpler, habit- ual, actor-critic methods and part of the striatum. We argue here for the normative appropriateness of an additional, but so far marginalized control system, associ- ated with episodic memory, and involving the hippocampus and medial temporal cortices. We analyze in depth a class of simple environments to show that episodic control should be useful in a range of cases characterized by complexity and in- ferential noise, and most particularly at the very early stages of learning, long before habitization has set in.

Experts have discovered an'impulsivity switch' in the brain that lets mammals suppress the urge to'jump the gun' and only act when the time is right. In lab experiments on mice, researchers found a brain area that's responsible for driving action and another that's responsible for suppressing that drive. Manipulating neurons, also known as nerve cells, in these areas can override our ability to control the urge to jump the gun and therefore trigger impulsive behaviour. Keeping the'impulsivity switch' on is how athletes stop themselves from running before the starting gun has fired, how dogs obey a command to resist a treat, or how lions in the wild can wait for the perfect moment to pounce on its prey. Keeping our'impulsivity switch' on is how athletes stop themselves from running before the starting gun has fired (file photo) 'We discovered a brain area responsible for driving action and another for suppressing that drive,' said study author Joe Paton, director of the Champalimaud Neuroscience Programme in Lisbon, Portugal.

Neuroscience studies of human decision-making abilities commonly involve subjects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subject's choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the model's internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.

Neuroscience studies of human decision-making abilities commonly involve subjects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subject's choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the model's internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.

They're often depicted as being lazy, but a new study suggests that teenagers are going through one of the best times to learn a new skill. Scientists have discovered increased activity in an area of the brain called the striatum in 17-20 year-olds, which boosts the way they learn from feedback. The findings suggest that adolescence is a unique life phase for increased feedback-learning performance. The researchers studied over 230 participants aged eight to 25. Each participant completed a feedback learning task, in which good performance was rewarded with positive feedback.

It is a question that has long stumped researchers. But now light has been shed on why boys are more at risk of autism. University of Iowa scientists believe they have collected the first ever evidence of a'protective effect' in females. Trials on mice showed males who had a known genetic cause of autism showed signs of being on the spectrum. This genetic deletion, or a missing stretch of DNA, plays a role in one in every 200 cases of autism spectrum disorder (ASD), experts claim. Figures suggest four boys are diagnosed with autism - which often causes sufferers to struggle with social interaction - to every one girl.