Conf-Profile: A Confidence-Driven Reasoning Paradigm for Label-Free User Profiling
Li, Yingxin, Zhao, Jianbo, Ren, Xueyu, Tang, Jie, You, Wangjie, Chen, Xu, Zhou, Kan, Feng, Chao, Ran, Jiao, Meng, Yuan, Wang, Zhi
–arXiv.org Artificial Intelligence
User profiling, as a core technique for user understanding, aims to infer structural attributes from user information. Large Language Models (LLMs) provide a promising avenue for user profiling, yet the progress is hindered by the lack of comprehensive benchmarks. To bridge this gap, we propose ProfileBench, an industrial benchmark derived from a real-world video platform, encompassing heterogeneous user data and a well-structured profiling taxonomy. However, the profiling task remains challenging due to the difficulty of collecting large-scale ground-truth labels, and the heterogeneous and noisy user information can compromise the reliability of LLMs. To approach label-free and reliable user profiling, we propose a Confidence-driven Profile reasoning framework Conf-Profile, featuring a two-stage paradigm. We first synthesize high-quality labels by leveraging advanced LLMs with confidence hints, followed by confidence-weighted voting for accuracy improvement and confidence calibration for a balanced distribution. The multiple profile results, rationales, and confidence scores are aggregated and distilled into a lightweight LLM. We further enhance the reasoning ability via confidence-guided unsupervised reinforcement learning, which exploits confidence for difficulty filtering, quasi-ground truth voting, and reward weighting. Experimental results demonstrate that Conf-Profile delivers substantial performance through the two-stage training, improving F1 by 13.97 on Qwen3-8B.
arXiv.org Artificial Intelligence
Sep-24-2025
- Country:
- Asia > China
- Guangdong Province > Shenzhen (0.04)
- Europe > Switzerland (0.04)
- North America > United States
- Oregon > Multnomah County > Portland (0.04)
- Asia > China
- Genre:
- Research Report > New Finding (0.48)
- Industry:
- Information Technology > Security & Privacy (1.00)
- Technology: