Evaluating Continual Test-Time Adaptation for Contextual and Semantic Domain Shifts