yt 1
Noisy Multi-Label Learning through Co-Occurrence-Aware Diffusion
Noisy labels often compel models to overfit, especially in multi-label classification tasks. Existing methods for noisy multi-label learning (NML) primarily follow a discriminative paradigm, which relies on noise transition matrix estimation or small-loss strategies to correct noisy labels. However, they remain substantial optimization difficulties compared to noisy single-label learning. In this paper, we propose a Co-Occurrence-Aware Diffusion (CAD) model, which reformulates NML from a generative perspective. We treat features as conditions and multilabels as diffusion targets, optimizing the diffusion model for multi-label learning with theoretical guarantees. Benefiting from the diffusion model's strength in capturing multi-object semantics and structured label matrix representation, we can effectively learn the posterior mapping from features to true multi-labels. To mitigate the interference of noisy labels in the forward process, we guide generation using pseudo-clean labels reconstructed from the latent neighborhood space, replacing original point-wise estimates with neighborhood-based proxies. In the reverse process, we further incorporate label co-occurrence constraints to enhance the model's awareness of incorrect generation directions, thereby promoting robust optimization. Extensive experiments on both synthetic (Pascal-VOC, MS-COCO) and real-world (NUS-WIDE) noisy datasets demonstrate that our approach outperforms state-of-the-art methods.
Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning
Davidov, Hen, Cohen, Nachshon, Kalinsky, Oren, Fairstein, Yaron, Kushilevitz, Guy, Yazdi, Ram, Rebeschini, Patrick
Large language models (LLMs) using chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising reasoning traces at each token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical reasoning and toxicity avoidance tasks support our theory and demonstrate improved selective accuracy over existing methods.
72dad95a24fae750f8ab1cb3dab5e58d-Supplemental-Conference.pdf
Forclassification tasks, asmentioned inboth ofthe experiments, we could easily adopt a human-machine collaboration framework: since our model is capable of conveying the prediction confidence along with the prediction itself, we could pass thecases where themodel islessassertivetohumans forfurther evaluation. Thistraitisespecially valuable for classification tasks with exceptionally imbalanced data,e.g., fraud detection, and ad click-through rate prediction, where the volume of one class could be orders of magnitude more than the other.
72dad95a24fae750f8ab1cb3dab5e58d-Paper-Conference.pdf
These additive-noise models areprimarily focusing onaccurately estimating theconditional mean E[y|x], while paying less attention to whether the noise distribution can accurately capture the uncertainty ofy given x. For this reason, they may not work well if the distribution ofy given x clearly deviates from the additive-noise assumption. For example, ifp(y|x) is multi-modal, which commonly happens when there are missing categorical covariates inx, then E[y|x] may not be close to any possible true values ofy given that specificx.