Baroni, Marco


Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks

arXiv.org Artificial Intelligence

Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb "dax," he or she can immediately understand the meaning of "dax twice" or "sing and dax." In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply "mix-and-match" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the "dax" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.



A Roadmap towards Machine Intelligence

arXiv.org Artificial Intelligence

The development of intelligent machines is one of the biggest unsolved challenges in computer science. In this paper, we propose some fundamental properties these machines should have, focusing in particular on communication and learning. We discuss a simple environment that could be used to incrementally teach a machine the basics of natural-language-based communication, as a prerequisite to more complex interaction with human users. We also present some conjectures on the sort of algorithms the machine should support in order to profitably learn from the environment.


The Concept Game: Better Commonsense Knowledge Extraction by Combining Text Mining and a Game with a Purpose

AAAI Conferences

Common sense collection has long been an important subfield of AI. This paper introduces a combined architecture for commonsense harvesting by text mining and a game with a purpose. The text miner module uses a seed set of known facts (sampled from ConceptNet) as training data and produces candidate commonsense facts mined from corpora. The game module taps humans' knowledge about the world by letting them play a simple slot-machine-like game. The proposed system allows us to collect significantly better commonsense facts than the state-of-the-art text miner alone, as shown experimentally for 5 rather different types of commonsense relations.