From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty