facebookresearch/CommAI-env

CommAI-env (Environment for Communication-based AI) is a platform for training and evaluating AI systems as described in A Roadmap towards Machine Intelligence. CommAI-env is a platform for training and testing an AI system, the Learner (coded in an arbitrary language of the system developer's choice), in a communication-based setup where it interacts via a bit-level interface with an Environment. The Environment asks the Learner to solve a number of communication-based Tasks, and assigns it a Reward for each task instance it successfully completes. The Learner is presented, in random order, with multiple instances of all tasks, and it has to solve as many of them as possible in order to maximize reward. Examples of tasks currently implemented include counting problems, tasks where the Learnes must memorize lists of items and answer questions about them, or follow navigation instructions through a text-based navigation scheme (see this document for detailed descriptions of the tasks). The set of tasks is open: we are constantly extending it, and we invite others to contribute.