Adapting Diffusion Models for Improved Prompt Compliance and Controllable Image Synthesis