Do DALL-E and Flamingo Understand Each Other?