On the Adversarial Robustness of Multi-Modal Foundation Models