A particularly problematic form of distribution shift comes from embedded power structures in data collection.

The ever increasing availability of cameras produces an endless stream of images.

SA TNet is a differentiable constraint solver with a custom backpropagation algorithm, which can be used as a layer in a deep-learning system. It is a promising proposal for bridging deep learning and logical reasoning. In fact, SA TNet has been successfully applied to learn, among others, the rules of a complex logical puzzle, such as Sudoku, just from input and output pairs where inputs are given as images. In this paper, we show how to improve the learning of SA TNet by exploiting symmetries in the target rules of a given but unknown logical puzzle or more generally a logical formula. We present SymSA TNet, a variant of SA T - Net that translates the given symmetries of the target rules to a condition on the parameters of SA TNet and requires that the parameters should have a particular parametric form that guarantees the condition. The requirement dramatically reduces the number of parameters to learn for the rules with enough symmetries, and makes the parameter learning of SymSA TNet much easier than that of SA TNet.

Multi-Task Learning (MTL) is a powerful learning paradigm to improve generalization performance via knowledge sharing.

This shows that local minibatching and device sampling are both beneficial for linearly speeding up communication. A few observations are in order.

We study the problem of robust estimation under heterogeneous corruption rates, where each sample may be independently corrupted with a known but non-identical probability. This setting arises naturally in distributed and federated learning, crowdsourcing, and sensor networks, yet existing robust estimators typically assume uniform or worst-case corruption, ignoring structural heterogeneity. For mean estimation for multivariate bounded distributions and univariate gaussian distributions, we give tight minimax rates for all heterogeneous corruption patterns. For multivariate gaussian mean estimation and linear regression, we establish the minimax rate for squared error up to a factor of $\sqrt{d}$, where $d$ is the dimension. Roughly, our findings suggest that samples beyond a certain corruption threshold may be discarded by the optimal estimators -- this threshold is determined by the empirical distribution of the corruption rates given.