This work tackles the problem of robust planning in non-stationary stochastic environments.

Second, we t = rt+1+ t+1x>t+1w x>t w , TD-errorforw (see (2)). This t is tobelarger t.

The solution in [5,4]requires aone-pass forward computation of marginal probabilities.

Understanding emerging behaviors of reinforcement learning (RL) agents may be difficult since such agents are often trained in complex environments using highly complex decision making procedures.

Thisoptimizationcan beperformedinparallelforeachs, keeping t 1 fixed.

In human pedagogy, teachers and students can interact adaptively to maximize communication efficiency. Theteacher adjusts herteaching method fordifferent students, and the student, after getting familiar with the teacher's instruction mechanism,caninfertheteacher'sintentiontolearnfaster.

Inthis paper, we consider Monte-Carlo planning in an environment with continuous state-action spaces, amuchlessunderstood problem withimportant applications in control and robotics.

Originating in psychology, $\textit{Theory of Mind}$ (ToM) has attracted significant attention across multiple research communities, especially logic, economics, and robotics. Most psychological work does not aim at formalizing those central concepts, namely $\textit{goals}$, $\textit{intentions}$, and $\textit{beliefs}$, to automate a ToM-based computational process, which, by contrast, has been extensively studied by logicians. In this paper, we offer a different perspective by proposing a computational framework viewed through the lens of game theory. On the one hand, the framework prescribes how to make boudedly rational decisions while maintaining a theory of mind about others (and recursively, each of the others holding a theory of mind about the rest); on the other hand, it employs statistical techniques and approximate solutions to retain computability of the inherent computational problem.

This paper introduces state abstraction for two-player zero-sum Markov games (TZMGs), where the payoffs for the two players are determined by the state representing the environment and their respective actions, with state transitions following Markov decision processes. For example, in games like soccer, the value of actions changes according to the state of play, and thus such games should be described as Markov games. In TZMGs, as the number of states increases, computing equilibria becomes more difficult. Therefore, we consider state abstraction, which reduces the number of states by treating multiple different states as a single state. There is a substantial body of research on finding optimal policies for Markov decision processes using state abstraction. However, in the multi-player setting, the game with state abstraction may yield different equilibrium solutions from those of the ground game. To evaluate the equilibrium solutions of the game with state abstraction, we derived bounds on the duality gap, which represents the distance from the equilibrium solutions of the ground game. Finally, we demonstrate our state abstraction with Markov Soccer, compute equilibrium policies, and examine the results.

AAAI Awards were presented in February at AAAI-24 in Vancouver, Canada. Each year, the Association for the Advancement of Artificial Intelligence recognizes its members, esteemed members of the AI community, and promising students, with the following awards and honors. The AAAI Award for Artificial Intelligence for the Benefit of Humanity recognizes the positive impacts of artificial intelligence to protect, enhance, and improve human life in meaningful ways with long-lived effects. The winner of this year's award is Milind Tambe (Harvard University/Google Research). Milind has been recognized for "ground-breaking applications of novel AI techniques to public safety and security, conservation, and public health, benefiting humanity on an international scale."