Enhancing Robot Navigation Policies with Task-Specific Uncertainty Management