Researchers propose game-based benchmark for AI's commonsense reasoning

In a paper accepted to last week's International Conference on Machine Learning, researchers at University College London and the University of Oxford propose an environment -- WordCraft -- to benchmark AI agents' commonsense reasoning capabilities. Based on Little Alchemy 2, a game that tasks players with mixing ingredients to create new items, they say WordCraft is both lightweight and built upon entities and relations inspired by real-world semantics. As the researchers note, personal assistants and household robots require agents that can learn quickly and generalize well to novel situations. That's likely not possible without the ability to reason using common sense and general knowledge about the world. For instance, an agent tasked with performing common household chores that hasn't seen a dirty ashtray would need to know a reasonable set of actions, including how to clean the ashtray and to avoid feeding it to a pet.