"Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs