Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting