Trump rushed into Iran crisis meeting as insider warns'strike within hours' 'Deeply concerned' King Charles backs Andrew investigation after royal's arrest and says the'law must take its course' Model agency boss who'scouted' victims for Epstein was secretly planning to testify against him... only to suddenly change his mind before meeting chillingly similar fate to notorious pedophile The monarchy has survived wars and countless crises... but this is why it may not survive Andrew's arrest - and why the rift at the heart of the family is about to get so much worse: ROBERT JOBSON Widower whose wife set herself on fire after alleged affair with married congressman finally breaks silence to reveal their texts... and heartbreaking video of her death Whereabouts of Andrew's ex-wife and daughters remain unknown as former prince is arrested over public misconduct claims Andrew'pushed through' appointment of Jeffery Epstein's fixer to board of Windsor Castle trust despite opposition Virginia Giuffre's family hail Andrew's arrest and say'he was never a prince' But countless women (and some husbands) are secretly getting it for thrilling sex side effects... risking a truly putrid complication FBI'has names and photos of people who may be masked suspect caught on surveillance video outside Nancy Guthrie's home' How DID Beatrice afford her 20s jet-set lifestyle? US assembles the most aerial firepower since Iraq War as Trump prepares to strike Iran'in just DAYS'... and president is'choosing between two devastating options of attack' The side-effects were unbearable and I swore off the drug forever. This is the simple diet that helped me shed the pounds... and I'm not alone. The Prince and Princess of Wales express support for King Charles' statement after Andrew's arrest Jason Bateman says he quit cocaine and alcohol to ease'tension' in his marriage Humiliating real reason Mia Goth left Shia LaBeouf: What'friends and lovers' are all saying behind his back... after Mardi Gras brawl What happens now Andrew Mountbatten-Windsor has been arrested? I went into early menopause in my 30s.

We investigate whether Large Language Models (LLMs) exhibit altruistic tendencies, and critically, whether their implicit associations and self-reports predict actual altruistic behavior. Using a multi-method approach inspired by human social psychology, we tested 24 frontier LLMs across three paradigms: (1) an Implicit Association Test (IAT) measuring implicit altruism bias, (2) a forced binary choice task measuring behavioral altruism, and (3) a self-assessment scale measuring explicit altruism beliefs. Our key findings are: (1) All models show strong implicit pro-altruism bias (mean IAT = 0.87, p < .0001), confirming models "know" altruism is good. (2) Models behave more altruistically than chance (65.6% vs. 50%, p < .0001), but with substantial variation (48-85%). (3) Implicit associations do not predict behavior (r = .22, p = .29). (4) Most critically, models systematically overestimate their own altruism, claiming 77.5% altruism while acting at 65.6% (p < .0001, Cohen's d = 1.08). This "virtue signaling gap" affects 75% of models tested. Based on these findings, we recommend the Calibration Gap (the discrepancy between self-reported and behavioral values) as a standardized alignment metric. Well-calibrated models are more predictable and behaviorally consistent; only 12.5% of models achieve the ideal combination of high prosocial behavior and accurate self-knowledge.

We thank the reviewers for their comments and actionable suggestions on improving the paper. We paraphrase some of the comments for brevity. We provide additional results on ResNet-18 for both CIFAR-10 and 100. Y our intuition is correct: for the baseline (i.e when's), the strong data augmentation prevents We will include a discussion on ROC and AUC curves for mixup in the final version (R1).

Reasoning language models have set state-of-the-art (SOTA) records on many challenging benchmarks, enabled by multi-step reasoning induced using reinforcement learning. However, like previous language models, reasoning models are prone to generating confident, plausible responses that are incorrect (hallucinations). Knowing when and how much to trust these models is critical to the safe deployment of reasoning models in real-world applications. To this end, we explore uncertainty quantification of reasoning models in this work. Specifically, we ask three fundamental questions: First, are reasoning models well-calibrated? Second, does deeper reasoning improve model calibration? Finally, inspired by humans' innate ability to double-check their thought processes to verify the validity of their answers and their confidence, we ask: can reasoning models improve their calibration by explicitly reasoning about their chain-of-thought traces? We introduce introspective uncertainty quantification (UQ) to explore this direction. In extensive evaluations on SOTA reasoning models across a broad range of benchmarks, we find that reasoning models: (i) are typically overconfident, with self-verbalized confidence estimates often greater than 85% particularly for incorrect responses, (ii) become even more overconfident with deeper reasoning, and (iii) can become better calibrated through introspection (e.g., o3-Mini and DeepSeek R1) but not uniformly (e.g., Claude 3.7 Sonnet becomes more poorly calibrated). Lastly, we conclude with important research directions to design necessary UQ benchmarks and improve the calibration of reasoning models.

When it comes to conspiracy theories, there are some pretty extreme ones out there. While some people insist the Earth is flat, others are certain the world is secretly ruled by reptilian humanoids. Now, a study has revealed that people who believe in these concepts are likely to be hugely overconfident. And it could go some way to explaining why it's impossible to try and change their minds. Analysis of eight studies has found a consistent pattern among people who believe in conspiracy theories – they tend to be overconfident in their cognitive abilities and significantly overestimate how much others agree with them.

As natural language becomes the default interface for human-AI interaction, there is a critical need for LMs to appropriately communicate uncertainties in downstream applications. In this work, we investigate how LMs incorporate confidence about their responses via natural language and how downstream users behave in response to LM-articulated uncertainties. We examine publicly deployed models and find that LMs are unable to express uncertainties when answering questions even when they produce incorrect responses. LMs can be explicitly prompted to express confidences, but tend to be overconfident, resulting in high error rates (on average 47%) among confident responses. We test the risks of LM overconfidence by running human experiments and show that users rely heavily on LM generations, whether or not they are marked by certainty. Lastly, we investigate the preference-annotated datasets used in RLHF alignment and find that humans have a bias against texts with uncertainty. Our work highlights a new set of safety harms facing human-LM interactions and proposes design recommendations and mitigating strategies moving forward.

Language models generate text based on successively sampling the next word. A decoding procedure based on nucleus (top-$p$) sampling chooses from the smallest possible set of words whose cumulative probability exceeds the probability $p$. In this work, we assess whether a top-$p$ set is indeed aligned with its probabilistic meaning in various linguistic contexts. We employ conformal prediction, a calibration procedure that focuses on the construction of minimal prediction sets according to a desired confidence level, to calibrate the parameter $p$ as a function of the entropy of the next word distribution. We find that OPT models are overconfident, and that calibration shows a moderate inverse scaling with model size.

The deployment of machine learning classifiers in high-stakes domains requires well-calibrated confidence scores for model predictions. In this paper we introduce the notion of variable-based calibration to characterize calibration properties of a model with respect to a variable of interest, generalizing traditional score-based metrics such as expected calibration error (ECE). In particular, we find that models with near-perfect ECE can exhibit significant miscalibration as a function of features of the data. We demonstrate this phenomenon both theoretically and in practice on multiple well-known datasets, and show that it can persist after the application of existing calibration methods. To mitigate this issue, we propose strategies for detection, visualization, and quantification of variable-based calibration error. We then examine the limitations of current score-based calibration methods and explore potential modifications. Finally, we discuss the implications of these findings, emphasizing that an understanding of calibration beyond simple aggregate measures is crucial for endeavors such as fairness and model interpretability.

Proper estimation of predictive uncertainty is fundamental in applications that involve critical decisions. Uncertainty can be used to assess the reliability of model predictions, trigger human intervention, or decide whether a model can be safely deployed in the wild. We introduce Fortuna, an open-source library for uncertainty quantification. Fortuna provides calibration methods, such as conformal prediction, that can be applied to any trained neural network to obtain calibrated uncertainty estimates. The library further supports a number of Bayesian inference methods that can be applied to deep neural networks written in Flax.