Inpainting is a learned interpolation technique that is based on generative modeling and used to populate masked or missing pieces in an image; it has wide applications in picture editing and retouching. Recently, inpainting started being used for watermark removal, raising concerns. In this paper we study how to manipulate it using our markpainting technique. First, we show how an image owner with access to an inpainting model can augment their image in such a way that any attempt to edit it using that model will add arbitrary visible information. We find that we can target multiple different models simultaneously with our technique. This can be designed to reconstitute a watermark if the editor had been trying to remove it. Second, we show that our markpainting technique is transferable to models that have different architectures or were trained on different datasets, so watermarks created using it are difficult for adversaries to remove. Markpainting is novel and can be used as a manipulation alarm that becomes visible in the event of inpainting.

Instead, we propose to integrate a generative inpainter into three representative attribution methods to remove an input feature. Compared to the original counterparts, our methods (1) generate more plausible counterfactual samples under the true data generating process; (2) are more robust to hyperparameter settings; and (3) localize objects more accurately. Our findings were consistent across both ImageNet and Places365 datasets and two different pairs of classifiers and inpainters. Explaining a classifier's outputs given a certain input is increasingly important, especially for life-critical applications (Doshi-V elez & Kim, 2017). A popular means for visually explaining an image classifier's decisions is an attribution map i.e. a heatmap that highlights the input pixels that are the evidence for and against the classification outputs (Montavon et al., 2018). To construct an attribution map, many methods approximate the attribution value of an input region by the classification probability change when that region is absent i.e. removed from the image. That is, most perturbation-based attribution methods implement the absence of an input feature by replacing it with (a) mean pixels; (b) random noise; or (c) blurred versions of the original content. While removing an input feature to measure its attribution is a principle method in causal reasoning, the existing removal (i.e. To combat these two issues, we propose to harness a state-of-the-art generative inpainting model (hereafter, an inpainter) to remove features from an input image and fill in with content that is plausible under the true data distribution. We test our approach on three representative attribution methods of Sliding-Patch (SP) (Zeiler & Fergus, 2014), LIME (Ribeiro et al., 2016), and Meaningful-Perturbation (MP) (Fong & V edaldi, 2017) across two large-scale datasets of ImageNet (Russakovsky et al., 2015) and Places365 (Zhou et al., 2017). For each dataset, we use a separate pair of pre-trained image classifiers and inpainters. Work done during CA's internship at Auburn University.