Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach

Large language models (LLMs) have marked a significant milestone in the advancement of natural language processing (Radford et al., 2019; Brown et al., 2020; Ouyang et al., 2022; Bubeck et al., 2023), showcasing remarkable capabilities in understanding and generating human-like text. However, one pressing issue for the LLMs is their propensity to hallucinate (Rawte et al., 2023) and generate misleading or entirely fabricated information that can significantly undermine their trustworthiness and reliability. The task of uncertainty estimation has then emerged to be an important problem that aims to determine the confidence levels of LLMs' outputs. While the problem of uncertainty estimation and calibration has seen considerable development within the general machine learning and deep learning domains (Abdar et al., 2021; Gawlikowski et al., 2023), we see less development in the domain of LLMs. One of the major challenges is the difference in the format of the output: while machine learning and deep learning typically involve fixed-dimensional outputs, natural language generation (NLG) tasks central to LLM applications require handling variable outputs that carry semantic meanings. Existing uncertainty estimation approaches for LLMs usually involve designing uncertainty metrics for their outputs. For black-box LLMs, these metrics are computed by examining aspects like the generated outputs' consistency, similarity, entropy, and other relevant characteristics (Lin et al., 2023; Manakul et al., 2023; Kuhn et al., 2023; Hou et al., 2023; Farquhar et al., 2024). Given the complexity of LLMs' underlying architectures, semantic information may be diluted when processing through self-attention mechanisms and during token encoding/decoding. To address this issue, a growing stream of literature argues that hidden layers' activation values within the LLMs offer insights into the LLMs' knowledge and confidence (Slobodkin et al., 2023; Ahdritz et al., 2024; Duan et al., 2024).