Using machine learning to address AI risk - Machine Intelligence Research Institute

At the EA Global 2016 conference, I gave a talk on "Using Machine Learning to Address AI Risk": It is plausible that future artificial general intelligence systems will share many qualities in common with present-day machine learning systems. If so, how could we ensure that these systems robustly act as intended? We discuss the technical agenda for a new project at MIRI focused on this question. The talk serves as a quick survey (for a general audience) of the kinds of technical problems we're working on under the "Alignment for Advanced ML Systems" research agenda. Included below is a version of the talk in blog post form.1 Actions are hard to evaluate 2.2. This talk is about a new research agenda aimed at using machine learning to make AI systems safe even at very high capability levels. I'll begin by summarizing the goal of the research agenda, and then go into more depth on six problem classes we're focusing on. The goal statement for this technical agenda is that we want to know how to train a smarter-than-human AI system to perform one or more large-scale, useful tasks in the world.