Why Robot Brains Need Symbols - Issue 67: Reboot

Humans can generalize a wide range of universals to arbitrary novel instances. They appear to do so in many areas of language (including syntax, morphology, and discourse) and thought (including transitive inference, entailments, and class-inclusion relationships). Advocates of symbol manipulation assume that the mind instantiates symbol-manipulating mechanisms including symbols, categories, and variables, and mechanisms for assigning instances to categories and representing and extending relationships between variables. This account provides a straightforward framework for understanding how universals are extended to arbitrary novel instances. Current eliminative connectionist models map input vectors to output vectors using the back-propagation algorithm (or one of its variants). To generalize universals to arbitrary novel instances, these models would need to generalize outside the training space. These models cannot generalize outside the training space. Therefore, current eliminative connectionist models cannot account for those cognitive phenomena that involve universals that can be freely extended to arbitrary cases. Richard Evans and Edward Grefenstette's recent paper at DeepMind, building on Joel Grus's blog post on the game Fizz-Buzz, follows remarkably similar lines, concluding that a canonical multilayer network was unable to solve the simple game on its own "because it did not capture the general, universally quantified rules needed to understand this task"--exactly what I said in 1998.