Fast Adaptive Test-Time Defense with Robust Features