Fast Proxies for LLM Robustness Evaluation