STAMP: Outlier-Aware Test-Time Adaptation with Stable Memory Replay