AsyncSwitch: Asynchronous Text-Speech Adaptation for Code-Switched ASR