Dynamic Programming Approximations for Partially Observable Stochastic Games

Partially observable stochastic games (POSGs) provide a rich mathematical framework for planning under uncertainty by a group of agents. However, this modeling advantage comes with a price, namely computation cost. Solving POSGs optimally quickly becomes intractable after a few decision cycles. Our main contribution is to provide bounded approximation techniques which enable us to scale POSG algorithms by several orders of magnitude. We study both the general POSGs and its cooperative counterpart DEC-POMDPs. Experiments on a number of problems confirm the scalability of our approach while still providing useful policies.