The assumption that data are independently and identically distributed (IID) is staple in statistical machine learning. It suggests that a hypothesis selected by an algorithm, after observing several training samples, should perform effectively on test samples from the same unknown distribution.

Neural Information Processing Systems http://nips.cc/

We formally guarantee that the distributed learners asymptotically achieve consensus which belongs to the set of stationary points of the bi-level optimization problem.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

The impossibility theorem of fairness is a foundational result in the algorithmic fairness literature. It states that outside of special cases, one cannot exactly and simultaneously satisfy all three common and intuitive definitions of fairness demographic parity, equalized odds, and predictive rate parity. This result has driven most works to focus on solutions for one or two of the metrics.

For more reliable comparison, we repeat experiments for100random seedsinstead of 10. "init tune" denotes tuningσ and choosing betweenN or U (see Figure 1 at the bottom); tuning isdone in the same wayasforotherhyperparameters. We will also add results of GCN supporting our conclusions (Table 115 and Figure 1). Note20 that in Table 1 of the submitted paper, forCOLORSand MNIST-75sp,21 ChebyGINs are equivalent to ChebyNets as described in Table 1 of22 theSupplementary material and elaborated onfollowing that table (see23 footnote3). In our model, the features are25 weighted by attention scores according to Eq. 3, so it is soft. In this26 case, the features indeed reduce their scale.