MapExRL: Human-Inspired Indoor Exploration with Predicted Environment Context and Reinforcement Learning

This work examines the question: How can a robot explore efficiently? We conduct a human user study to gain insights into effective exploration strategies. These insights, in turn, inform the design of our reinforcement learning-based exploration policy, leveraging global map predictions and other environmental contexts and enabling state-of-the-art performance. Abstract -- Path planning for robotic exploration is challenging, requiring reasoning over unknown spaces and anticipating future observations. Efficient exploration requires selecting budget-constrained paths that maximize information gain. Despite advances in autonomous exploration, existing algorithms still fall short of human performance, particularly in structured environments where predictive cues exist but are underutilized. Guided by insights from our user study, we introduce MapExRL, which improves robot exploration efficiency in structured indoor environments by enabling longer-horizon planning through reinforcement learning (RL) and global map predictions. Our framework generates global map predictions from the observed map, which our policy utilizes, along with the prediction uncertainty, estimated sensor coverage, frontier distance, and remaining distance budget, to assess the strategic long-term value of frontiers. By leveraging multiple frontier scoring methods and additional context, our policy makes more informed decisions at each stage of the exploration. We evaluate our framework on a real-world indoor map dataset, achieving up to an 18.8% improvement over the strongest state-of-the-art baseline, with even greater gains compared to conventional frontier-based algorithms. This work involved human subjects or animals in its research.