Grounding Language Plans in Demonstrations Through Counterfactual Perturbations