Rao-Blackwellized POMDP Planning

Abstract--Partially Observable Markov Decision Processes (POMDPs) provide a structured framework for decision-making under uncertainty, but their application requires efficient belief updates. Sequential Importance Resampling Particle Filters (SIRPF), also known as Bootstrap Particle Filters, are commonly used as belief updaters in large approximate POMDP solvers, but they face challenges such as particle deprivation and high computational costs as the system's state dimension grows. To address these issues, this study introduces Rao-Blackwellized POMDP (RB-POMDP) approximate solvers and outlines generic methods to apply Rao-Blackwellization in both belief updates and online planning. POMCPOW (left) and RB-POMCPOW (right) Tree Structure Comparison. Moreover, as Partially Observable Markov Decision Processes (POMDPs) the system's effective dimension grows, a substantial increase are a powerful mathematical framework for modeling in the number of particles may be required to maintain decision-making under uncertainty where an agent operates performance, resulting in high computational costs (e.g. Rao-Blackwellized Particle Filtering (RBPF) offer a promising POMDPs have been widely applied to various domains such solution to address some of these limitations of the SIRPF.