Quantifying the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data