Enhancing Robot Navigation Policies with Task-Specific Uncertainty Managements