Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks.

In response to the efficiency problem, recent studies show that dense PLMs can be replaced with sparse subnetworks without hurting the performance. Such subnetworks can be found in three scenarios: 1) the fine-tuned PLMs, 2) the raw PLMs and then fine-tuned in isolation, and even inside 3) PLMs without any parameter fine-tuning. However, these results are only obtained in the in-distribution (ID) setting.

As the curation of data for machine learning becomes increasingly automated, dataset tampering is a mounting threat.

We show that deconfounding the similarity metrics increases the resolution of detecting functionally similar neural networks across domains.