Animal behavior is driven by multiple brain regions working in parallel with distinct control policies. We present a biologically plausible model of off-policy reinforcement learning in the basal ganglia, which enables learning in such an architecture. The model accounts for action-related modulation of dopamine activity that is not captured by previous models that implement on-policy algorithms. In particular, the model predicts that dopamine activity signals a combination of reward prediction error (as in classic models) and "action surprise," a measure of how unexpected an action is relative to the basal ganglia's current policy. In the presence of the action surprise term, the model implements an approximate form of Q-learning.

Researchers have found that genetically and pharmacologically restoring the normal activity of the brain circuit improved anorexia, opening the possibility of developing a treatment strategy for affected individuals in the future. Researchers at Baylor College of Medicine, Louisiana State University and collaborating institutions has discovered that abnormal activity in a particular brain circuit underlies anorexia in an animal model of the condition. Genetically and pharmacologically restoring the normal activity of the brain circuit improved the condition, opening the possibility of developing a treatment strategy for affected individuals in the future. Anorexia has no approved treatment, and the underlying causes is unclear. The study was recently published in Nature Neuroscience.

DeepMind just recently published a paper detailing how a newly developed type of reinforcement learning could potentially explain how reward pathways within the human brain operate. As reported by NewScientist, the machine learning training method is called distributional reinforcement learning and the mechanisms behind it seem to plausibly explain how dopamine is released by neurons within the brain. Neuroscience and computer science have a long history together. As far back as 1951, Marvin Minksy used a system of rewards and punishments to create a computer program capable of solving a maze. Minksy was inspired by the work of Ivan Pavlov, a physiologist who demonstrated that dogs could learn through a series of rewards and punishments.

Developments in artificial intelligence often draw inspiration from how humans think, but now AI has turned the tables to teach us about how brains learn. Will Dabney at tech firm DeepMind in London and his colleagues have found that a recent development in machine learning called distributional reinforcement learning also provides a new explanation for how the reward pathways in the brain work. These pathways govern our response to pleasurable events and are mediated by neurons that release the brain chemical dopamine. "Dopamine in the brain is a type of surprise signal," says Dabney. "When things turn out better than expected, more dopamine gets released."