Weight matrix in [18] is W(l) = W(l,1) kW(l,2).

Graph Neural Networks (GNNs) are widely applied to graph learning problems such as node classification. When scaling up the underlying graphs of GNNs to a larger size, we are forced to either train on the complete graph and keep the full graph adjacency and node embeddings in memory (which is often infeasible) or mini-batch sample the graph (which results in exponentially growing computational complexities with respect to the number of GNN layers). Various sampling-based and historical-embedding-based methods are proposed to avoid this exponential growth of complexities. However, none of these solutions eliminates the linear dependence on graph size. This paper proposes a sketch-based algorithm whose training time and memory grow sublinearly with respect to graph size by training GNNs atop a few compact sketches of graph adjacency and node embeddings. Based on polynomial tensor-sketch (PTS) theory, our framework provides a novel protocol for sketching non-linear activations and graph convolution matrices in GNNs, as opposed to existing methods that sketch linear weights or gradients in neural networks. In addition, we develop a locality sensitive hashing (LSH) technique that can be trained to improve the quality of sketches. Experiments on large-graph benchmarks demonstrate the scalability and competitive performance of our Sketch-GNNs versus their full-size GNN counterparts.

The hardware employed4 consisted of 24 Intel(R) Xeon(R) Gold 5317 CPUs @ 3.00GHz, 8 modules of 32GB memory (with a5 speed of 3200MT/s), and 2 NVIDIAA40 GPUs with 48GB of memory each (NVIDIAUNIX x86_646 Kernel Module 510.108.03, CUDA version 11.6, cuDNN version 8.3).7 A.2 Network Architecture8 For the program synthesizing stage, the structure of the I/O encoder is elaborated in Table 1, where9 we employ dk1 dk2-s-do Conv to denote the 2D convolution with kernel size dk1 dk2, stride s, and10 output channel do. Additionally, BN refers to batch normalization [8], and di-do Linear denotes the11 fully-connected layer with input feature di and output feature do. The I/O encoder utilizes residual12 networks [7] and takes I/O pair with size 5 5 3 as inputs. To improve candidate programs through environmental contexts, the decoder's structure is elaborated14 in Table 2. Here, we utilize do-hGATv2Conv to represent the dynamic graph attention variant [1]15 with output channel do and multiple attention heads h, and do-nl denotes the nl layered bi-directional16 LSTM with output feature do.

Program synthesis aims to automatically generate an executable program that conforms to the given specification. Recent advancements have demonstrated that deep neural methodologies and large-scale pretrained language models are highly proficient in capturing program semantics. For robot programming, prior works have facilitated program synthesis by incorporating global environments. However, the assumption of acquiring a comprehensive understanding of the entire environment is often excessively challenging to achieve. In this work, we present a framework that learns to synthesize a program by rectifying potentially erroneous code segments, with the aid of partially observed environments. To tackle the issue of inadequate attention to partial observations, we propose to first learn an environment embedding space that can implicitly evaluate the impacts of each program token based on the precondition. Furthermore, by employing a graph structure, the model can aggregate both environmental and syntactic information flow and furnish smooth program rectification guidance. Extensive experimental evaluations and ablation studies on the partially observed VizDoom domain authenticate that our method offers superior generalization capability across various tasks and greater robustness when encountering noises.

In this section we prove the theoretical results around the dual curriculum game and use these results to show approximation bounds for our methods, given that they have reached a Nash equilibrium (NE). The first theorem is the main result that allows us to analyze dual curriculum games. The high-level result says that the NE of a dual curriculum game are approximate NE of the base game from the perspective of any of the individual players, or from the perspective of the joint strategy. Let Bbe the maximum difference between U1t and U2t, and let (π,θ1,θ2) be a NE for G. Then (π,pθ1 + (1 p)θ2) is an approximate NE for the base game with either teacher or for a teacher optimizing their joint objective. More precisely, it is a 2Bp(1 p)-approximate NE when Ut = pU1t + (1 p)U2t, a 2B(1 p)-approximate NE when Ut = U1t, and a 2Bp-approximate NE when Ut = U2t. At a high level, this is true because, for low values of p, the best-response strategies for the individual players can be thought of as approximate-best response strategies for the joint-player, and vis-versa. Since the Nash Equilibrium consists of each of the players playing their own best response, they must be playing an approximate best response for the joint-player. We provide a formal proof below: Proof. Let B be the maximum difference between U1t and U2t, and let (π,θ1,θ2) be a Nash Equilibrium for G. Then consider pθ1 + (1 p)θ2 as a strategy in the base game for the joint player pU1t + (1 p)U2t.

No-reference image quality assessment (NR-IQA) aims to quantify how humans perceive visual distortions of digital images without access to their undistorted references. NR-IQA models are extensively studied in computational vision, and are widely used for performance evaluation and perceptual optimization of man-made vision systems. Here we make one of the first attempts to examine the perceptual robustness of NR-IQA models. Under a Lagrangian formulation, we identify insightful connections of the proposed perceptual attack to previous beautiful ideas in computer vision and machine learning. We test one knowledgedriven and three data-driven NR-IQA methods under four full-reference IQA models (as approximations to human perception of just-noticeable differences). Through carefully designed psychophysical experiments, we find that all four NRIQA models are vulnerable to the proposed perceptual attack. More interestingly, we observe that the generated counterexamples are not transferable, manifesting themselves as distinct design flows of respective NR-IQA methods.

Tabular data (or tables) are the most widely used data format in machine learning (ML). However, ML models often assume the table structure keeps fixed in training and testing. Before ML modeling, heavy data cleaning is required to merge disparate tables with different columns.