We denote observed variables with gray color and latent variables with white color. Firstly, we introduce the concept of an uncontrolled style factor . Why do confident examples encourage content-style isolation? Calculate the loss using Eq. 1 and update networks; Output: The inference networks and classifier heads q It's essential to understand that although data augmentation cannot control all style factors, it still offers the benefit of "partial isolation". This approach, therefore, ensures that styles changes don't affect the derived content representation Calculate the loss using Eq. 2 and update networks; Output: The inference networks and classifier heads q Finally, confident and unlabeled examples are used to train the models based on the MixMatch algorithm.

Appendix: How does a Neural Network's Architecture Impact its Robustness to Noisy Labels? In this section, we include additional experimental results for the predictive power in (a) representations from randomly initialized models (Appendix A.1), (b) representations learned under different We first evaluate the predictive power of randomly initialized models (a.k.a., untrained models), and Notice that lower test MAPE denotes better test performance.Model T est MAPE Random Trained Max-sum GNN 12.74 0.57 0.37 0.08 In previous experiments (section 4.2), we have shown that the predictive power in well-aligned MAE, is more helpful in learning more predictive representations under smaller noise ratios. The predictive in the representations grows as the mutual information between the noisy labels and original clean labels increases for models well-aligned with the target function. Clean Labels (DwC) and further measure the predictive power in representations learned by DwC. Table 6: Test accuracy (%) on CIF AR-10 with flipped label noise .

We denote observed variables with gray color and latent variables with white color. Firstly, we introduce the concept of an uncontrolled style factor . Why do confident examples encourage content-style isolation? Calculate the loss using Eq. 1 and update networks; Output: The inference networks and classifier heads q It's essential to understand that although data augmentation cannot control all style factors, it still offers the benefit of "partial isolation". This approach, therefore, ensures that styles changes don't affect the derived content representation Calculate the loss using Eq. 2 and update networks; Output: The inference networks and classifier heads q Finally, confident and unlabeled examples are used to train the models based on the MixMatch algorithm.

We are grateful to the reviewers for their time and their thoughtful comments, which we believe will improve the paper. We first clarify the comparison with DivideMix and then address all individual comments below. Following the same approach as DivideMix, the accuracy for ELR+ is the same or even higher (e.g. We will explain all of this in our revision, including possible limitations as suggested by Reviewer 3. The memorization effect is not new to the community. We believe that it may be due to a reduction in confirmation bias.

Appendix: How does a Neural Network's Architecture Impact its Robustness to Noisy Labels? In this section, we include additional experimental results for the predictive power in (a) representations from randomly initialized models (Appendix A.1), (b) representations learned under different We first evaluate the predictive power of randomly initialized models (a.k.a., untrained models), and Notice that lower test MAPE denotes better test performance.Model T est MAPE Random Trained Max-sum GNN 12.74 0.57 0.37 0.08 In previous experiments (section 4.2), we have shown that the predictive power in well-aligned MAE, is more helpful in learning more predictive representations under smaller noise ratios. The predictive in the representations grows as the mutual information between the noisy labels and original clean labels increases for models well-aligned with the target function. Clean Labels (DwC) and further measure the predictive power in representations learned by DwC. Table 6: Test accuracy (%) on CIF AR-10 with flipped label noise .

Deep learning models rely heavily on large volumes of labeled data to achieve high performance. However, real-world datasets often contain noisy labels due to human error, ambiguity, or resource constraints during the annotation process. Instance-dependent label noise (IDN), where the probability of a label being corrupted depends on the input features, poses a significant challenge because it is more prevalent and harder to address than instance-independent noise. In this paper, we propose a novel hybrid framework that combines self-supervised learning using SimCLR with iterative pseudo-label refinement to mitigate the effects of IDN. The self-supervised pre-training phase enables the model to learn robust feature representations without relying on potentially noisy labels, establishing a noise-agnostic foundation. Subsequently, we employ an iterative training process with pseudo-label refinement, where confidently predicted samples are identified through a multistage approach and their labels are updated to improve label quality progressively. We evaluate our method on the CIFAR-10 and CIFAR-100 datasets augmented with synthetic instance-dependent noise at varying noise levels. Experimental results demonstrate that our approach significantly outperforms several state-of-the-art methods, particularly under high noise conditions, achieving notable improvements in classification accuracy and robustness. Our findings suggest that integrating self-supervised learning with iterative pseudo-label refinement offers an effective strategy for training deep neural networks on noisy datasets afflicted by instance-dependent label noise.

Forgetting presents a significant challenge during incremental training, making it particularly demanding for contemporary AI systems to assimilate new knowledge in streaming data environments. To address this issue, most approaches in Continual Learning (CL) rely on the replay of a restricted buffer of past data. However, the presence of noise in real-world scenarios, where human annotation is constrained by time limitations or where data is automatically gathered from the web, frequently renders these strategies vulnerable. In this study, we address the problem of CL under Noisy Labels (CLN) by introducing Alternate Experience Replay (AER), which takes advantage of forgetting to maintain a clear distinction between clean, complex, and noisy samples in the memory buffer. The idea is that complex or mislabeled examples, which hardly fit the previously learned data distribution, are most likely to be forgotten. To grasp the benefits of such a separation, we equip AER with Asymmetric Balanced Sampling (ABS): a new sample selection strategy that prioritizes purity on the current task while retaining relevant samples from the past. Through extensive computational comparisons, we demonstrate the effectiveness of our approach in terms of both accuracy and purity of the obtained buffer, resulting in a remarkable average gain of 4.71% points in accuracy with respect to existing loss-based purification strategies. Code is available at https://github.com/aimagelab/mammoth.

Deep neural network models for IMU sensor-based human activity recognition (HAR) that are trained from controlled, well-curated datasets suffer from poor generalizability in practical deployments. However, data collected from naturalistic settings often contains significant label noise. In this work, we examine two in-the-wild HAR datasets and DivideMix, a state-of-the-art learning with noise labels (LNL) method to understand the extent and impacts of noisy labels in training data. Our empirical analysis reveals that the substantial domain gaps among diverse subjects cause LNL methods to violate a key underlying assumption, namely, neural networks tend to fit simpler (and thus clean) data in early training epochs. Motivated by the insights, we design VALERIAN, an invariant feature learning method for in-the-wild wearable sensor-based HAR. By training a multi-task model with separate task-specific layers for each subject, VALERIAN allows noisy labels to be dealt with individually while benefiting from shared feature representation across subjects. We evaluated VALERIAN on four datasets, two collected in a controlled environment and two in the wild.

In many classification problems, collecting massive clean-annotated data is not easy, and thus a lot of researches have been done to handle data with noisy labels. Most recent state-of-art solutions for noisy label problems are built on the small-loss strategy which exploits the memorization effect. While it is a powerful tool, the memorization effect has several drawbacks. The performances are sensitive to the choice of a training epoch required for utilizing the memorization effect. In addition, when the labels are heavily contaminated or imbalanced, the memorization effect may not occur in which case the methods based on the small-loss strategy fail to identify clean labeled data. We introduce a new method called INN(Integration with the Nearest Neighborhoods) to refine clean labeled data from training data with noisy labels. The proposed method is based on a new discovery that a prediction pattern at neighbor regions of clean labeled data is consistently different from that of noisy labeled data regardless of training epochs. The INN method requires more computation but is much stable and powerful than the small-loss strategy. By carrying out various experiments, we demonstrate that the INN method resolves the shortcomings in the memorization effect successfully and thus is helpful to construct more accurate deep prediction models with training data with noisy labels.