A.1 Proof Sketch We first introduce the following lemma: Lemma 1. In general, it is hard to develop a convergence rate for objective values. By Theorem 5, we can also show the superiority of FedSubAvg over FedAvg. We then assume that FedSubAvg always activates all the clients at the beginning of each communication round and then uses the parameters maintained by a few selected clients to generate the next-round parameter. It is clear that this update scheme is equivalent to the original.

The global model for federated optimization typically contains a large and sparse embedding layer, while each client's local data tend to interact with part of features, updating only a small submodel with the feature-related embedding vectors.

Replogle experiment additional details Each model was optimized with the Adam optimizer for 150,000 steps with batch size 512, learning rate 0.0003, and weight decay 1E-6.

Research supporting this publication conducted while authors were employed at insitro 37th Conference on Neural Information Processing Systems (NeurIPS 2023).

We sincerely appreciate the reviewers for their careful reading, constructive questions and suggestions. First, we address questions appeared at least twice. We write P1, P2 for paragraph reference, and Rx for reviewers. That is, they only consider when the representations are precisely equal. To the best of our knowledge, our work is the first to incorporate continuous similarity into designing GNN.

In this paper, we aim to answer three key open questions: (i) whether and how brain networks are cross-correlated at different time scales with long-range dependencies ("long-memory" process,

Existing strategies for finite-armed stochastic bandits mostly depend on a parameter of scale that must be known in advance. Sometimes this is in the form of a bound on the payoffs, or the knowledge of a variance or subgaussian parameter. The notable exceptions are the analysis of Gaussian bandits with unknown mean and variance by Cowan et al. [2015] and of uniform distributions with unknown support [Cowan and Katehakis, 2015]. The results derived in these specialised cases are generalised here to the non-parametric setup, where the learner knows only a bound on the kurtosis of the noise, which is a scale free measure of the extremity of outliers.