Are Slepian-Wolf Rates Necessary for Distributed Parameter Estimation?

There are two main different setups for statistical learning: centralized learning and distributed learning. In the centralized learning, which has been studied extensively, all data is available at a centralized location. In the distributed learning, data is stored in multiple terminals. The distributed learning setup has attracted significant recent research interests as the data involved in learning is increasingly large in volume and might be stored in multiple terminals [1], [2], [3], [4]. For the distributed learning, each terminal either has a few observations about all variables, or has full knowledge about a subset of variables (all observations about a subset of variables). The first scenario is relatively easier since each terminal can still make its own local inference without even communicating with each other, while communication between terminals is essential for the second scenario. In this paper, we focus on the more challenging second scenario. In particular, we consider a distributed parameter estimation problem.