Diffusing in Someone Else's Shoes: Robotic Perspective Taking with Diffusion