AlignDiff: Learning Physically-Grounded Camera Alignment via Diffusion