Rethinking Garment Conditioning in Diffusion-based Virtual Try-On