Towards Map-Agnostic Policies for Adaptive Informative Path Planning