LLM Stability: A detailed analysis with some surprises