Feed-Forward Source-Free Latent Domain Adaptation via Cross-Attention