From teaching machines to teaching people: an interview with Michael Littman

Welcome back to AI Pioneers, our series of in-depth conversations with leading figures in artificial intelligence. In this interview, we speak with Michael Littman about the evolution of reinforcement learning, the role of creativity and culture in how humans and machines learn, and the challenges generative AI poses for education. We also explore his approach to science communication, his use of music to convey computer science concepts, and his unexpected connection to the first multi-media email attachment . You've been in the field of reinforcement learning for decades. How did you get into it, and which questions have you found most interesting across your career? I was drawn to reinforcement learning when I was a college student in the mid to late '80s. At that time, people were just starting to talk about neural networks, and the machine learning techniques being used were supervised, which is very focused on the input and output. I didn't find that so interesting - I wanted to know how to make them behave. So, I created my own reinforcement-learning system to address the game of noughts and crosses, or tic-tac-toe. It was a small enough game that I could run it on my little computer at home, but it made me address some interesting questions. For example, what does it mean to get feedback? And how do you use feedback at the end of the game to know what you should have done at the beginning of the game? I started to explore this topic without knowing that it was reinforcement learning. It wasn't until 1988, when I got out of college, that I discovered the link. I was working in a research lab with a researcher named David Ackley, who had done work on similar kinds of models. He used to be a student of Geoff Hinton, so he was very familiar with neural networks and very interested in behavior. I explained to him what I was playing with and why I found it so interesting, and he said "oh, that's called reinforcement learning, Rich Sutton just published a paper on this - let's get him to come and give a talk."