Historical Test-time Prompt Tuning for Vision Foundation Models