In-Context Policy Adaptation via Cross-Domain Skill Diffusion