On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots