Intelligence is not Artificial

Summarizing, there are four desiderata that one would like to see in A.I. systems, if they have to compare well with human (or just animal) brains: meta-learning, learning by demonstration ("few-shot learning"), transfer learning and multi-task learning. Meta-learning is particularly relevant in the case of reinforcement learning. It is obvious that reinforcement learning is highly unnatural. DeepMind's AlphaGo and OpenAi Five need to learn from scratch via a huge number of trials. Animals, instead, use built-in or acquired "meta-skills" to learn new tasks in just a few trials. Modern computational theory of meta-learning (learning how to learn) dates back at least to the 1990s, when Schmidhuber published the manifesto "Simple Principles of Metalearning" (1996), followed by his student Sepp Hochreiter ("Learning to Learn Using Gradient Descent", 2001), and by Nicolas Schweighofer and Kenji Doya at Japan's ATR ("Meta-learning in Reinforcement Learning", 2001). Examples of "deep" meta-learning systems of the new generation are: RL Square by Pieter Abbeel's student Yan Duan at UC Berkeley, based on Schulman's TRPO ("RL Square: Fast Reinforcement Learning via Slow Reinforcement Learning", 2016); the "model-agnostic meta-learning" (MAML) of Sergey Levine's student Chelsea Finn at UC Berkeley ("Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks", 2017); Marcel Binz's thesis at KTH Royal Institute of Technology ("Learning Goal-Directed Behaviour", 2017); Jane Wang's "deep meta-reinforcement learning" at DeepMind ("Learning to Reinforcement Learn", 2017); and OpenAI's Reptile, developed by Alex Nichol and John Schulman, a generalization of Finn's MAML ("On First-Order Meta-Learning Algorithms", 2018). DeepMind's neuroscientist Matthew Botvinick believes that the latter could be a model for how our brain learns: the dopamine system trains another part of the brain, the prefrontal cortex, to operate as its own free-standing learning system ("Prefrontal Cortex as a Meta-reinforcement Learning System", 2018).