The pretraining-finetuning paradigm has gained widespread adoption in vision tasks and other fields. However, the finetuning phase still requires high-quality annotated samples.

Inlabel-noiselearning,thenoise transitionmatrix,bridgingtheclassposteriorfor noisy and clean data, has been widely exploited to learnstatistically consistent classifiers.

Theorem1.Let Pi represent the distribution of all seen training samples (including all previous Agoodtrade-offbetween themodel'scomplexityandgeneralization performance, observedfrom Eq. (12), is allowing each component to learn the underlying data distribution of a unique target set. By satisfying the ideal selection process (Eq.(22) of the paper) and also consideringthateachcomponent Gtfinishedthetrainingon Mkt atTkt,weassumethatthedynamic 4 expansion modelG can be seen as a single modelh trained on all previously learnt memories Maximal Interfered Retrieval (MIR), [1] is one of 5 themostpopular memory-based approaches, whichusesamemory bufferwithasample selection criterion. Since Pi would involve several underlying data distributions as the number of training steps (i) increases, the diversity in the memory plays an important role to ensure a tight GB in Eq.(15). G be single model which consists of a classifierh HandaVAEmodelv. M be a memory buffer updated at the training stepTi. Figure 1: The learning process of the proposed ODDL-S, which consists of three phases.

A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous work often depend on task boundary and i.i.d.

The pretraining-finetuning paradigm has gained widespread adoption in vision tasks and other fields. However, the finetuning phase still requires high-quality annotated samples. To overcome this challenge, the concept of active finetuning has emerged, aiming to select the most appropriate samples for model finetuning within a limited budget.

Fairness and robustness are critical elements of Trustworthy AI that need to be addressed together. Fairness is about learning an unbiased model while robustness is about learning from corrupted data, and it is known that addressing only one of them may have an adverse affect on the other. In this work, we propose a sample selection-based algorithm for fair and robust training. To this end, we formulate a combinatorial optimization problem for the unbiased selection of samples in the presence of data corruption. Observing that solving this optimization problem is strongly NP-hard, we propose a greedy algorithm that is efficient and effective in practice. Experiments show that our method obtains fairness and robustness that are better than or comparable to the state-of-the-art technique, both on synthetic and benchmark real datasets. Moreover, unlike other fair and robust training baselines, our algorithm can be used by only modifying the sampling step in batch selection without changing the training algorithm or leveraging additional clean data.

Modern deep networks achieve impressive performance when trained on large, clean, and carefully curated datasets. In realistic data mining scenarios, however, labels come from heterogeneous sources such as crowdsourcing, weak supervision, or heuristic rules and are therefore noisy [18, 3]. Human annotation errors, ambiguous images, and domain shifts all contribute to mislabeled or outlier samples that can harm generalization. In image classification, for example, web-scale datasets often contain wrong tags or near-duplicate images with conflicting labels; in user-generated content analysis, spam or off-topic posts corrupt the training distribution. Data cleaning is widely recognized as crucial [15] but is typically performed before model training using hand-crafted rules or separate anomaly detectors [9, 16]. This two-stage design has two drawbacks: (i) it requires domain expertise or extra supervision to specify cleaning rules and thresholds; (ii) it decouples cleaning from model optimization, so the decisions do not directly leverage discriminative feedback from the task model. Some high-loss samples may still be informative "hard cases," whereas others are truly corrupted and should be discarded. We explore a different paradigm: can the model learn which samples to trust during training, treating data cleaning as an integral, differentiable part of optimization?