Feature Visualization (FV) is a widely used technique for interpreting concepts learned by Deep Neural Networks (DNNs), which synthesizes input patterns that maximally activate a given feature. Despite its popularity, the trustworthiness of FV explanations has received limited attention. We introduce Gradient Slingshots, a novel method that enables FV manipulation without modifying model architecture or significantly degrading performance.

Current vision systems are trained on huge datasets, and these datasets come with costs: curation is expensive, they inherit human biases, and there are concerns over privacy and usage rights. To counter these costs, interest has surged in learning from cheaper data sources, such as unlabeled images. In this paper, we go a step further and ask if we can do away with real image datasets entirely, by learning from procedural noise processes. We investigate a suite of image generation models that produce images from simple random processes. These are then used as training data for a visual representation learner with a contrastive loss. In particular, we study statistical image models, randomly initialized deep generative models, and procedural graphics models. Our findings show that it is important for the noise to capture certain structural properties of real data but that good performance can be achieved even with processes that are far from realistic. We also find that diversity is a key property for learning good representations.

In light of the recent widespread adoption of AI systems, understanding the internal information processing of neural networks has become increasingly critical.

Recall measures the fraction ofcommon samples that are retrievedascorrect common class, while specificity measures thefraction ofprivatesamples thatarenotretrieved. Fig. S1(b) shows the sensitivity ofγ, where γ is the rough boundary for splitting positive and negative in adaptive filling. For the cosine similarity of two ℓ2-normalized features, the similarity value is limited from 1to1, where higher value indicates higher similarity. Suchself-supervisedlearning methods encourage the consistency between two augmentations of one image. The display images for source prototypes are chosen by finding the nearest source instance of the prototype.

Tiny-ImageNet isasmall subset of ImageNet dataset, containing 100,000 training images, 10,000 validation images, and 10,000 testing images separated in 200 different classes, dimensionsofwhichare64 64pixels. Here,anapproximate featureprobability q(Z) is introduced to approximate the true feature probabilityp(Z). The additional results are illustrated in Figure 1. We provide additional feature visualization under various adversarial attack methods including NRF in Figure 1-5 (CIFAR-10, SVHN, and Tiny-ImageNet are utilized). Moreover,thedistilled features still include therobustand brittle information eveninthefailed attack examples.