Can LLMs Reconcile Knowledge Conflicts in Counterfactual Reasoning

Large Language Models have been shown to contain extensive world knowledge in their parameters, enabling impressive performance on many knowledge intensive tasks. However, when deployed in novel settings, LLMs often encounter situations where they must integrate parametric knowledge with new or unfamiliar information. In this work, we explore whether LLMs can combine knowledge in-context with their parametric knowledge through the lens of counterfactual reasoning. Through synthetic and real experiments in multi-hop reasoning problems, we show that LLMs generally struggle with counterfactual reasoning, often resorting to exclusively using their parametric knowledge. Moreover, we show that simple post-hoc finetuning can struggle to instill counterfactual reasoning ability - often leading to degradation in stored parametric knowledge. Ultimately, our work reveals important limitations of current LLM's abilities to re-purpose parametric knowledge in novel settings. Benchmarks like NaturalQuestions and HotpotQA have driven progress on recall-based and multi-hop reasoning, but they primarily evaluate a model's ability to regurgitate stored facts or compose chains of parametric knowledge without new external inputs (Y ang et al., 2018; Kwiatkowski et al., 2019). In contrast, many real-world scenarios require LLMs to integrate their pretrained knowledge with novel or hypothetical information provided at inference time. For example, consider a counterfactual query: "If Paris were located in Italy, in which country would the Eiffel T ower stand?"