Modern reinforcement learning has been conditioned by at least three dogmas. The first is the environment spotlight, which refers to our tendency to focus on modeling environments rather than agents. The second is our treatment of learning as finding the solution to a task, rather than adaptation. The third is the reward hypothesis, which states that all goals and purposes can be well thought of as maximization of a reward signal. These three dogmas shape much of what we think of as the science of reinforcement learning. While each of the dogmas have played an important role in developing the field, it is time we bring them to the surface and reflect on whether they belong as basic ingredients of our scientific paradigm. In order to realize the potential of reinforcement learning as a canonical frame for researching intelligent agents, we suggest that it is time we shed dogmas one and two entirely, and embrace a nuanced approach to the third.

Neurosurgeon Michael Fehlings specializes in complex spine surgery and is the Vice Chair Research for the Department of Surgery at the University of Toronto and Co-Director of the University of Toronto Spine Program. We caught up with him to learn more about where the use of Artificial Intelligence (AI) in neurosurgery could be heading in the future. What inspired you to specialize in neuroscience and spinal cord injuries? I first became intrigued with serious disorders of the brain because my grandfather sustained multiple strokes. I saw the impact that brain disease had on him and it robbed him of his language. When I went to medical school I was fascinated with brain disorders and I felt that neurosurgery provided practical solutions to help people like my grandfather.

Almost 200 years ago, the renowned U.S. naturalist John James Audubon hid a decaying pig carcass under a pile of brush to test vultures' sense of smell. When the birds overlooked the pig—while one flocked to a nearly odorless stuffed deer skin—he took it as proof that they rely on vision, not smell, to find their food. His experiment cemented a commonly held idea. Despite later evidence that vultures and a few specialized avian hunters use odors after all, the dogma that most birds aren't attuned to smell endured. Now, that dogma is being eroded by findings on birds' behavior and molecular hardware, two of which were published just last month. One showed storks home in on the smell of freshly mowed grass; another documented scores of functional olfactory receptors in multiple bird species. Researchers are realizing, says evolutionary biologist Scott Edwards of Harvard University, that “olfaction has a lot of impact on different aspects of bird biology.” Forty years ago, when ethologist Floriano Papi proposed that homing pigeons find their way back to a roost by sniffing out its chemical signature, his colleagues scoffed at the idea. They pointed out that birds have several other keen senses to guide them, including sight and, in the case of pigeons and some other species, a magnetic sense. “By then, biological textbooks already stated unequivocally that birds have little to no sense of smell, and many people still believe it—even scientists,” says Danielle Whittaker, a chemical ecologist at Michigan State University. Still, contrary evidence was already accumulating. In the 1960s, ornithologist Kenneth Stager found vultures were attracted to boxes with a carcass hidden inside and fans that vented the odors—as long as this bait wasn't too decomposed, as was likely the case in Audubon's experiment. Researchers also found that albatrosses, shearwaters, and some other seabirds find their fish prey by detecting a chemical released by the plankton the fish eat. But these birds, forced to navigate many kilometers across a featureless sea, seemed exceptional. In 2008, “You were part of the dark side if you talked about birds using olfaction,” recalls Martin Wikelski, an ecologist at the Max Planck Institute for Ornithology. That year, though, a graduate student at his institute, molecular ecologist Silke Steiger, analyzed nine bird genomes from across the avian family tree and uncovered many genes for olfactory receptors—proteins in the nasal passages that bind to odors and relay a signal to the brain. In species that don't rely much on smell (humans are an example), these genes often mutate and become nonfunctional. But the researchers confirmed that many of the birds' olfactory genes were intact. What's more, they found that the number of these genes correlated with the size of a bird species' olfactory bulb, the brain's smell center—further evidence that the receptors were functional. The genomes in that study were incomplete, however. Last month, Christopher Balakrishnan, an evolutionary biologist at East Carolina University, and graduate student Robert Driver examined some of the best available bird genomes and for some species found many more olfactory genes. Their analysis of genomes from a hummingbird, emu, chicken, zebra finch, and a tropical fruit eater called a manakin revealed scores of new olfactory receptors, they reported on 28 June in the journal Integrative and Comparative Biology . That the emu has so many of these genes excites Whittaker, because this bird sits near the base of the bird family tree. “This result suggests that the ancestor to all birds must have had a very diverse set of olfactory receptor genes as well,” she says. Smell must have been important to birds from the beginning, and comparisons of their olfactory receptor genes today confirm it remains so. Balakrishnan and Driver found that one diverse set of receptors unique to birds has split into multiple types specific to different bird lineages. That suggests these genes evolved rapidly as the birds diversified. Natural selection may have honed the genes to perform crucial tasks. Wikelski and colleagues saw bird smell in action after they were inspired by a question from a curious primary school student. During an outreach program at a school in Radolfzell, Germany, the student asked the scientists how the local population of European white storks found their way to freshly cut meadows, where their insect and rodent prey were most exposed. To find out, Wikelski piloted his plane in circles to observe a flock of 70 storks on sunny spring and summer days. Even when the storks couldn't see or hear the mowing, he and his colleagues noted, they homed in on mowed fields upwind of them, as if drawn to the smell of the cut grass. To confirm the suspicion, the team sprayed cut-grass smell—a mix of three volatile chemicals—onto fields that hadn't been mowed recently. The storks came flocking, the team reported on 18 June in Scientific Reports . The work “shows very clearly that these birds rely exclusively on their sense of smell to make foraging decisions,” Whittaker says. Other bird species may also respond to “calls” from injured plants, recent evidence shows. Two European birds, the great tit and the blue tit, locate insects that are attacking pine trees by detecting the volatile chemicals the stressed trees release, ecologist Elina Mäntylä of the Biology Centre of the Czech Academy of Sciences and colleagues reported in the September 2020 issue of Ecology and Evolution . All these results show bird olfaction “should not be ignored,” Mäntylä says. Driver adds that they might also point to a new form of natural pest control, in which farmers or foresters could treat threatened flora with chemicals that entice birds to come and gobble up invasive insects. Other studies suggest olfaction might guide social interactions between birds. Whittaker's team has focused on preen oil, which birds secrete from a gland at the base of the tail and rub onto their feathers. The oil's chemical composition reveals the bird's species, sex, aggressiveness, and reproductive state. Females produce much more of these odorous chemicals, Whittaker and her colleagues reported in January in the Journal of Chemical Ecology , suggesting they depend more on odors to communicate, lacking the flashy feathers and songs that males rely on. Use of these cues is “likely widespread,” says Steiger, now at the German chemical company BASF SE, “but simply not yet investigated well enough.” That's changing fast, as studies of bird olfaction expand into new species. Published papers on the topic have doubled every decade since 1992, reaching 80 this past year. The field is, belatedly, putting Audubon's misconception to rest and acknowledging that birds—champions of flight, vision, and song—have another power as well.

Capcom has finally decided to release its action role-playing game "Dragon's Dogma: Dark Arisen" for Microsoft and Sony's newer gaming consoles. The Japanese video game developer announced the good news this Monday via the official Twitter account of the game. Fans immediately expressed their happiness over the re-release of the 5-year-old video game. "This is my favorite game of all time! I played it numerous times on Xbox 360. I now have a chance to enjoy it again on the PS4. Tank you," Twitter user @Travo9486 wrote.