Uncovering what neural nets "see" with FlashTorch

Feature visualisation is an active area of research which aims to understand how neural networks perceive images, by exploring ways in which we can look "through their eyes". It has emerged and evolved in response to an increasing desire to make neural networks more interpretable to humans. Earliest works include analysing what neural networks are paying attention to within input images. For example, image-specific class saliency maps visualise regions within an input image that contribute the most to the corresponding output, by calculating the gradient of a class output with respect to the input image via backpropagation (more on saliency maps later in the post). Another strand of technique in feature visualisation is activation maximisation.