Without any knowledge of the trigger, it is difficult to distinguish or recover benign DNNs from backdoored ones.

The rapid development of Large Language Models (LLMs) has been pivotal in advancing AI, with pre-trained LLMs being adaptable to diverse downstream tasks through fine-tuning. Federated learning (FL) further enhances fine-tuning in a privacy-aware manner by utilizing clients' local data through in-situ computation, eliminating the need for data movement. However, fine-tuning LLMs, given their massive scale of parameters, poses challenges for clients with constrained and heterogeneous resources in FL.

We adopted the existing code from Amersfoort et al. to train the DUQ models. DIP/INR and the proposed manifold consistency, it can still be challenging to avoid trivial solutions. However, given the large solution space, this often leads to unrealistic images. For this experiment, we used the CelebA faces dataset and considered the baldness attribute. Figure 1: Examples of counterfactuals generated for the baldness and age attributes using DISC.

Most popular benchmarks for comparing LLMs rely on alimited set ofprompt templates, which may not fully capture the LLMs' abilities and can affect the reproducibility ofresults onleaderboards. Manyrecent worksempirically verify prompt sensitivity and advocate for changes in LLM evaluation.