Goto

Collaborating Authors

 littman


From teaching machines to teaching people: an interview with Michael Littman

AIHub

Welcome back to AI Pioneers, our series of in-depth conversations with leading figures in artificial intelligence. In this interview, we speak with Michael Littman about the evolution of reinforcement learning, the role of creativity and culture in how humans and machines learn, and the challenges generative AI poses for education. We also explore his approach to science communication, his use of music to convey computer science concepts, and his unexpected connection to the first multi-media email attachment . You've been in the field of reinforcement learning for decades. How did you get into it, and which questions have you found most interesting across your career? I was drawn to reinforcement learning when I was a college student in the mid to late '80s. At that time, people were just starting to talk about neural networks, and the machine learning techniques being used were supervised, which is very focused on the input and output. I didn't find that so interesting - I wanted to know how to make them behave. So, I created my own reinforcement-learning system to address the game of noughts and crosses, or tic-tac-toe. It was a small enough game that I could run it on my little computer at home, but it made me address some interesting questions. For example, what does it mean to get feedback? And how do you use feedback at the end of the game to know what you should have done at the beginning of the game? I started to explore this topic without knowing that it was reinforcement learning. It wasn't until 1988, when I got out of college, that I discovered the link. I was working in a research lab with a researcher named David Ackley, who had done work on similar kinds of models. He used to be a student of Geoff Hinton, so he was very familiar with neural networks and very interested in behavior. I explained to him what I was playing with and why I found it so interesting, and he said "oh, that's called reinforcement learning, Rich Sutton just published a paper on this - let's get him to come and give a talk."





ExplainableReinforcementLearningviaModel Transforms

Neural Information Processing Systems

Understanding emerging behaviors of reinforcement learning (RL) agents may be difficult since such agents are often trained in complex environments using highly complex decision making procedures.



IterativeTeacher-AwareLearning

Neural Information Processing Systems

In human pedagogy, teachers and students can interact adaptively to maximize communication efficiency. Theteacher adjusts herteaching method fordifferent students, and the student, after getting familiar with the teacher's instruction mechanism,caninfertheteacher'sintentiontolearnfaster.



A Computable Game-Theoretic Framework for Multi-Agent Theory of Mind

arXiv.org Artificial Intelligence

Originating in psychology, $\textit{Theory of Mind}$ (ToM) has attracted significant attention across multiple research communities, especially logic, economics, and robotics. Most psychological work does not aim at formalizing those central concepts, namely $\textit{goals}$, $\textit{intentions}$, and $\textit{beliefs}$, to automate a ToM-based computational process, which, by contrast, has been extensively studied by logicians. In this paper, we offer a different perspective by proposing a computational framework viewed through the lens of game theory. On the one hand, the framework prescribes how to make boudedly rational decisions while maintaining a theory of mind about others (and recursively, each of the others holding a theory of mind about the rest); on the other hand, it employs statistical techniques and approximate solutions to retain computability of the inherent computational problem.


Approximate State Abstraction for Markov Games

arXiv.org Artificial Intelligence

This paper introduces state abstraction for two-player zero-sum Markov games (TZMGs), where the payoffs for the two players are determined by the state representing the environment and their respective actions, with state transitions following Markov decision processes. For example, in games like soccer, the value of actions changes according to the state of play, and thus such games should be described as Markov games. In TZMGs, as the number of states increases, computing equilibria becomes more difficult. Therefore, we consider state abstraction, which reduces the number of states by treating multiple different states as a single state. There is a substantial body of research on finding optimal policies for Markov decision processes using state abstraction. However, in the multi-player setting, the game with state abstraction may yield different equilibrium solutions from those of the ground game. To evaluate the equilibrium solutions of the game with state abstraction, we derived bounds on the duality gap, which represents the distance from the equilibrium solutions of the ground game. Finally, we demonstrate our state abstraction with Markov Soccer, compute equilibrium policies, and examine the results.