Visualizing Representations: Deep Learning and Human Beings - colah's blog


Imagine training a neural network and watching its representations wander through this space. You can see how your representations compare to other "landmark" representations from past experiments. If your model's first layer representation is in the same place a really successful model's was during training, that's a good sign! If it's veering off towards a cluster you know had too high learning rates, you know you should lower it. This can give us qualitative feedback during neural network training.