Converting text to images for product discovery

Generative adversarial networks (GANs), which were first introduced in 2014, have proven remarkably successful at generating synthetic images. A GAN consists of two networks, one that tries to produce convincing fakes, and one that tries to distinguish fakes from real examples. The two networks are trained together, and the competition between them can converge quickly on a useful generative model. In a paper that was accepted to IEEE's Winter Conference on Applications of Computer Vision, we describe a new use of GANs to generate examples of clothing that match textual product descriptions. The idea is that a shopper could use a visual guide to refine a text query until it reliably retrieved the product for which she or he was looking.