Clothes and color extraction with Generative Adversarial Network

So how can we solve this problem? I tried to replace the background of the original image with a solid color manually and realized that with this kind of input the model produces much better results. But how can this job be automated?