Revisit the Stability of Vanilla Federated Learning Under Diverse Conditions