Spatio-Temporal Deep Graph Infomax

MUTUALINFORMATION MAXIMIZATION Deep InfoMax (DIM, Hjelm et al., 2019) is a recent approach for unsupervised representation learning that derives embeddings by maximizing the mutual information between the output of an encoder and local patches of the input. DIM builds on Mutual Information Neural Information (MINE, Belghazi et al., 2018), which formulates an estimate Î(X; Y) for the mutual information between random variables X, Y using neural networks. These estimates are obtained by training a classifier (a.k.a, the discriminator or statistics network) to distinguish between samples from the joint distribution and the product of marginals. DIM applies this approach to representation learning by training both the encoder and the discriminator to maximize the mutual information between the random variables corresponding to local input patches and the embeddings. Deep Graph Infomax (DGI) extends this representation learning technique to non-temporal graphs, finding node embeddings that maximize the mutual information between local patches of the graph and summaries of the entire graph. Here, we build on these methods and propose a representation learning technique for spatiotemporal graphs. Furthermore, unlike in previous work, we evaluate our embeddings in the regression rather than classification setting.