Foundation models are trained on increasingly immense and opaque datasets.

Deep learning methods have boosted the adoption of NLP systems in real-life applications.

Hinton et al. first propose the concept of distillation, where a lightweight student model

Figure 1 compares the result of MI attacks using model predictions alone (PredMI) and explanation-aware MI attacks using model predictions and explanations together (ExpMI).