Single Mesh Diffusion Models with Field Latents for Texture Generation