Cross-Domain Continual Learning via CLAMP