Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception? In Section 4.1, we briefly introduced how humans annotate the reconstructed images for different datasets. In the supplementary material, we have included a graphical user interface (GUI) that was utilized by the annotators. Figure 1 displays the GUI, where (A) and (B) were specifically designed for annotating different datasets. To minimize the influence of subjective bias, we use a relatively objective formulation: whether the reconstructed image can be correctly labeled.

A.1 MONet To segment each w hframe Ft into No object representations, MONet uses a recurrent attention network to obtain No attention masks Ati [0,1]w h for i = 1,...,No that represent the probability of each pixel in Ft belonging to the i-th object, with This attention network is coupled with a component VAE with latents zti Rd for i= 1,...,No that reconstructs Ati Ft, the i-th object in the image. The latent posterior distribution q(zt|Ft,Ati)is a diagonal Gaussian with mean µti, and we use µti as the representation of the i-th object. When these representations are fed into the transformer, we use a linear projection to map the raw object/word embeddings, which lie in Rd, to a vector in RdNH, where NH is the number of selfattention heads. This step is necessary as generally the latent dimensionality of MONet, d, is less than NH whereas a transformer expects the embedding size to be divisible by NH. A.2 Self-supervised training Recall in the main text that we wrote the auxiliary self-supervised loss as auxiliary loss = X A comparison of these losses and the masking schemes is given in Figure 4. We also tested a few variations of the contrastive loss inspired by literature and tested all combinations of variations.

Neural networks have achieved success in a wide array of perceptual tasks but often fail at tasks involving both perception and higher-level reasoning. On these more challenging tasks, bespoke approaches (such as modular symbolic components, independent dynamics models or semantic parsers) targeted towards that specific type of task have typically performed better. The downside to these targeted approaches, however, is that they can be more brittle than general-purpose neural networks, requiring significant modification or even redesign according to the particular task at hand. Here, we propose a more general neural-network-based approach to dynamic visual reasoning problems that obtains state-of-the-art performance on three different domains, in each case outperforming bespoke modular approaches tailored specifically to the task. Our method relies on learned object-centric representations, self-attention and self-supervised dynamics learning, and all three elements together are required for strong performance to emerge. The success of this combination suggests that there may be no need to trade off flexibility for performance on problems involving spatio-temporal or causal-style reasoning. With the right soft biases and learning objectives in a neural network we may be able to attain the best of both worlds.

Adaptive gradient methods have shown excellent performances for solving many machine learning problems. Although multiple adaptive gradient methods were recently studied, they mainly focus on either empirical or theoretical aspects and also only work for specific problems by using some specific adaptive learning rates. Thus, it is desired to design a universal framework for practical algorithms of adaptive gradients with theoretical guarantee to solve general problems. To fill this gap, we propose a faster and universal framework of adaptive gradients (i.e., SUPER-ADAM) by introducing a universal adaptive matrix that includes most existing adaptive gradient forms. Moreover, our framework can flexibly integrate the momentum and variance reduced techniques. In particular, our novel framework provides the convergence analysis support for adaptive gradient methods under the nonconvex setting. In theoretical analysis, we prove that our SUPER-ADAM algorithm can achieve the best known gradient (i.e., stochastic first-order oracle (SFO)) complexity of O( 3) for finding an -stationary point of nonconvex optimization, which matches the lower bound for stochastic smooth nonconvex optimization. In numerical experiments, we employ various deep learning tasks to validate that our algorithm consistently outperforms the existing adaptive algorithms.

A.1 Graph Spectrum Consider an undirected graph G =( V,E)whose adjacency matrix is symmetric. Following notations in Section 2, we denote the eigendecomposition of the normalized graph adjacency and Laplacian matrices respectively as A = UMU > and L = VNV >, where M = diag(µ1,,µn), |µ1| |µ2| | µn|, N = diag( 1,, n), 0= 1 < 2 n, and U,V are the matrices of corresponding eigenvectors. We also immediately have A2 = UM 2U> = U U>, f = |µf |2. Intuitively, since L = I A, the leading eigenvalues µ1,µ2, of A correspond to the smallest of those 1, 2, of L. These eigenvalues are known as the low-frequency spectrum of the graph that correlates to graph connectivity. Specially, 2 > 0if and only if the graph is connected, which is our case. Similarly, small values of µf and large values of i represent the high frequency part of the graph. Graph spectrum is a graph invariant despite the status of node labels. We plot the spectrum computed by A2Prop of 7 heterophilous graphs and 2 homophilous graphs in Figure 4, which shows the leading k eigenvalues µ1,µ2,,µk. The figure indicates the importance of high-frequency information in graphs under heterophily. For appropriately large heterophilous graphs penn94, arxiv-year, genius, and twitch-gamers, the spectrum converges slowly to 0. In other words, even high-frequency parts with large f still exhibit relatively large eigenvalues compared to the dominant µ1. In contrast, the homophilous protein and reddit are significant in low frequency of a few eigenvalues. Hence, during the iterative propagation, the topology information carried by these high-frequency components can be preserved to enhance performance.