This leads to a new MU paradigm, termed prune first, then unlearn, which infuses a sparse model prior into the unlearning process.

Have you retrained or moved careers due to your previous career path being at risk of an artificial intelligence takeover? Please include as much detail as possible. Did you have a dream profession that you have decided not to pursue because of fears it will be thwarted by AI? Optional Please include as much detail as possible.

For problems with a very large vocabulary size, the embedding and the softmax matrices can account for more than half of the model size. For instance, the bigLSTM model achieves great performance on the One-Billion-Word (OBW) dataset with around 800k vocabulary, and its word embedding and softmax matrices use more than 6GBytes space, and are responsible for over 90% of the model parameters. In this paper, we propose GroupReduce, a novel compression method for neural language models, based on vocabulary-partition (block) based low-rank matrix approximation and the inherent frequency distribution of tokens (the power-law distribution of words).

CURE4Rec covers four aspects, i.e., unlearning Com-10

For problems with a very large vocabulary size, the embedding and the softmax matrices can account for more than half of the model size. For instance, the bigLSTM model achieves great performance on the One-Billion-Word (OBW) dataset with around 800k vocabulary, and its word embedding and softmax matrices use more than 6GBytes space, and are responsible for over 90% of the model parameters. In this paper, we propose GroupReduce, a novel compression method for neural language models, based on vocabulary-partition (block) based low-rank matrix approximation and the inherent frequency distribution of tokens (the power-law distribution of words).

"blur the distributions": As Wasserstein barycenter adjusts the support, blurring is more likely for Euclidean V anilla averaging, in contrast, fails to fine-tune despite trying numerous settings of optimization hyperparameters. Also, Fig 1, shows similar gains for data-free post-processing in case of structured pruning (as in Sec 5.2). V anilla average fails to retrain. Results shown are mean std. "there could possibly be more competent baselines": The'constraint' of performing this without sharing of sensitive training data arises in many applications, "improvement over vanilla averaging is very marginal": We respectfully disagree.

We thank the reviewers for their detailed and insightful comments.

This leads to a new MU paradigm, termed prune first, then unlearn, which infuses a sparse model prior into the unlearning process.