Diffusion-Based Electromagnetic Inverse Design of Scattering Structured Media