To what extent is the success of deep visualization due to the training? Could we do deep visualization using untrained, random weight networks? To address this issue, we explore new and powerful generative models for three popular deep visualization tasks using untrained, random weight convolutional neural networks. First we invert representations in feature spaces and reconstruct images from white noise inputs. The reconstruction quality is statistically higher than that of the same method applied on well trained networks with the same architecture.

I think this is a very interesting finding, especially considering that this was claimed not to be possible in Gatys et al. (Texture Synthesis Using Convolutional Neural Networks). Your paper makes a comment about how your improved scaling of the gradients for the different layers is the difference maker. However, it would be interesting to see further study into this with experiments showing exactly how the results get worse as the scaling is artificially made less appropriate. I also think these results could have impact on application. Training time is one thing, but I think it is ever more useful not to have to store the weights.

To what extent is the success of deep visualization due to the training? Could we do deep visualization using untrained, random weight networks? To address this issue, we explore new and powerful generative models for three popular deep visualization tasks using untrained, random weight convolutional neural networks. First we invert representations in feature spaces and reconstruct images from white noise inputs. The reconstruction quality is statistically higher than that of the same method applied on well trained networks with the same architecture.