Off-Policy Evaluation with Online Adaptation for Robot Exploration in Challenging Environments