We present an active mapping system that could plan for long-horizon exploration goals and short-term actions with a 3D Gaussian Splatting (3DGS) representation. Existing methods either did not take advantage of recent developments in multimodal Large Language Models (LLM) or did not consider challenges in localization uncertainty, which is critical in embodied agents. We propose employing multimodal LLMs for long-horizon planning in conjunction with detailed motion planning using our information-based algorithm. By leveraging high-quality view synthesis from our 3DGS representation, our method employs a multimodal LLM as a zero-shot planner for long-horizon exploration goals from the semantic perspective. We also introduce an uncertainty-aware path proposal and selection algorithm that balances the dual objectives of maximizing the information gain for the environment while minimizing the cost of localization errors. Experiments conducted on the Gibson and Habitat-Matterport 3D datasets demonstrate state-of-the-art results of the proposed method.

Informative path planning (IPP) applied to bathymetric mapping allows AUVs to focus on feature-rich areas to quickly reduce uncertainty and increase mapping efficiency. Existing methods based on Bayesian optimization (BO) over Gaussian Process (GP) maps work well on small scenarios but they are short-sighted and computationally heavy when mapping larger areas, hindering deployment in real applications. To overcome this, we present a 2-layered BO IPP method that performs non-myopic, real-time planning in a tree search fashion over large Stochastic Variational GP maps, while respecting the AUV motion constraints and accounting for localization uncertainty. Our framework outperforms the standard industrial lawn-mowing pattern and a myopic baseline in a set of hardware in the loop (HIL) experiments in an embedded platform over real bathymetry.

Navigating and localizing in partially observable, stochastic environments with magnetic anomalies presents significant challenges, especially when balancing the accuracy of state estimation and the stability of localization. Traditional approaches often struggle to maintain performance due to limited localization updates and dynamic conditions. This paper introduces a multi-objective global path planner for magnetic anomaly navigation (MagNav), which leverages entropy maps to assess spatial frequency variations in magnetic fields and identify high-information areas. The system generates paths toward these regions by employing a potential field planner, enhancing active localization. Hardware experiments demonstrate that the proposed method significantly improves localization stability and accuracy compared to existing active localization techniques. The results underscore the effectiveness of this method in reducing localization uncertainty and highlight its adaptability to various gradient-based navigation maps, including topographical and underwater depth-based environments.

Localization in GPS-denied environments is critical for autonomous systems, and traditional methods like SLAM have limitations in generalizability across diverse environments. Magnetic-based navigation (MagNav) offers a robust solution by leveraging the ubiquity and unique anomalies of external magnetic fields. This paper proposes a real-time uncertainty-aware motion planning algorithm for MagNav, using onboard magnetometers and information-driven methodologies to adjust trajectories based on real-time localization confidence. This approach balances the trade-off between finding the shortest or most energy-efficient routes and reducing localization uncertainty, enhancing navigational accuracy and reliability. The novel algorithm integrates an uncertainty-driven framework with magnetic-based localization, creating a real-time adaptive system capable of minimizing localization errors in complex environments. Extensive simulations and real-world experiments validate the method, demonstrating significant reductions in localization uncertainty and the feasibility of real-time implementation. The paper also details the mathematical modeling of uncertainty, the algorithmic foundation of the planning approach, and the practical implications of using magnetic fields for localization. Future work includes incorporating a global path planner to address the local nature of the current guidance law, further enhancing the method's suitability for long-duration operations.

Multi-object tracking (MOT) methods have seen a significant boost in performance recently, due to strong interest from the research community and steadily improving object detection methods. The majority of tracking methods follow the tracking-by-detection (TBD) paradigm, blindly trust the incoming detections with no sense of their associated localization uncertainty. This lack of uncertainty awareness poses a problem in safety-critical tasks such as autonomous driving where passengers could be put at risk due to erroneous detections that have propagated to downstream tasks, including MOT. While there are existing works in probabilistic object detection that predict the localization uncertainty around the boxes, no work in 2D MOT for autonomous driving has studied whether these estimates are meaningful enough to be leveraged effectively in object tracking. We introduce UncertaintyTrack, a collection of extensions that can be applied to multiple TBD trackers to account for localization uncertainty estimates from probabilistic object detectors. Experiments on the Berkeley Deep Drive MOT dataset show that the combination of our method and informative uncertainty estimates reduces the number of ID switches by around 19\% and improves mMOTA by 2-3%. The source code is available at https://github.com/TRAILab/UncertaintyTrack

This paper presents MULAN-WC, a novel multi-robot 3D reconstruction framework that leverages wireless signal-based coordination between robots and Neural Radiance Fields (NeRF). Our approach addresses key challenges in multi-robot 3D reconstruction, including inter-robot pose estimation, localization uncertainty quantification, and active best-next-view selection. We introduce a method for using wireless Angle-of-Arrival (AoA) and ranging measurements to estimate relative poses between robots, as well as quantifying and incorporating the uncertainty embedded in the wireless localization of these pose estimates into the NeRF training loss to mitigate the impact of inaccurate camera poses. Furthermore, we propose an active view selection approach that accounts for robot pose uncertainty when determining the next-best views to improve the 3D reconstruction, enabling faster convergence through intelligent view selection. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of our framework in theory and in practice. Leveraging wireless coordination and localization uncertainty-aware training, MULAN-WC can achieve high-quality 3d reconstruction which is close to applying the ground truth camera poses. Furthermore, the quantification of the information gain from a novel view enables consistent rendering quality improvement with incrementally captured images by commending the robot the novel view position. Our hardware experiments showcase the practicality of deploying MULAN-WC to real robotic systems.

We propose an autonomous exploration algorithm designed for decentralized multi-robot teams, which takes into account map and localization uncertainties of range-sensing mobile robots. Virtual landmarks are used to quantify the combined impact of process noise and sensor noise on map uncertainty. Additionally, we employ an iterative expectation-maximization inspired algorithm to assess the potential outcomes of both a local robot's and its neighbors' next-step actions. To evaluate the effectiveness of our framework, we conduct a comparative analysis with state-of-the-art algorithms. The results of our experiments show the proposed algorithm's capacity to strike a balance between curbing map uncertainty and achieving efficient task allocation among robots.

We present MOTLEE, a distributed mobile multi-object tracking algorithm that enables a team of robots to collaboratively track moving objects in the presence of localization error. Existing approaches to distributed tracking make limiting assumptions regarding the relative spatial relationship of sensors, including assuming a static sensor network or that perfect localization is available. Instead, we develop an algorithm based on the Kalman-Consensus filter for distributed tracking that properly leverages localization uncertainty in collaborative tracking. Further, our method allows the team to maintain an accurate understanding of dynamic objects in the environment by realigning robot frames and incorporating frame alignment uncertainty into our object tracking formulation. We evaluate our method in hardware on a team of three mobile ground robots tracking four people. Compared to previous works that do not account for localization error, we show that MOTLEE is resilient to localization uncertainties, enabling accurate tracking in distributed, dynamic settings with mobile tracking sensors.

We present DyFOS, an active perception method that dynamically finds optimal states to minimize localization uncertainty while avoiding obstacles and occlusions. We consider the scenario where a perception-denied rover relies on position and uncertainty measurements from a viewer robot to localize itself along an obstacle-filled path. The position uncertainty from the viewer's sensor is a function of the states of the sensor itself, the rover, and the surrounding environment. To find an optimal sensor state that minimizes the rover's localization uncertainty, DyFOS uses a localization uncertainty prediction pipeline in an optimization search. Given numerous samples of the states mentioned above, the pipeline predicts the rover's localization uncertainty with the help of a trained, complex state-dependent sensor measurement model (a probabilistic neural network). Our pipeline also predicts occlusion and obstacle collision to remove undesirable viewer states and reduce unnecessary computations. We evaluate the proposed method numerically and in simulation. Our results show that DyFOS is faster than brute force yet performs on par. DyFOS also yielded lower localization uncertainties than faster random and heuristic-based searches.

Autonomous inspection tasks necessitate effective path-planning mechanisms to efficiently gather observations from points of interest (POI). However, localization errors commonly encountered in urban environments can introduce execution uncertainty, posing challenges to the successful completion of such tasks. To tackle these challenges, we present IRIS-under uncertainty (IRIS-U^2), an extension of the incremental random inspection-roadmap search (IRIS) algorithm, that addresses the offline planning problem via an A*-based approach, where the planning process occurs prior the online execution. The key insight behind IRIS-U^2 is transforming the computed localization uncertainty, obtained through Monte Carlo (MC) sampling, into a POI probability. IRIS-U^2 offers insights into the expected performance of the execution task by providing confidence intervals (CI) for the expected coverage, expected path length, and collision probability, which becomes progressively tighter as the number of MC samples increase. The efficacy of IRIS-U^2 is demonstrated through a case study focusing on structural inspections of bridges. Our approach exhibits improved expected coverage, reduced collision probability, and yields increasingly-precise CIs as the number of MC samples grows. Furthermore, we emphasize the potential advantages of computing bounded sub-optimal solutions to reduce computation time while still maintaining the same CI boundaries.