deviate
Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Large language models (LLMs) frequently generate hallucinations--content that deviates from factually inaccurate or deviates from provided context--posing challenges for diagnosis. However, diagnosing the causes of hallucination is challenging due to the complex interplay of underlying causes. This paper introduces a framework to systematically understand the sources of hallucination behavior in large language models. Our key insight is that hallucinations arise when more frequent but non-factual associations outweigh faithful ones. Through theoretical and empirical analyses, we demonstrate that decoder-only transformers effectively function as subsequence embedding models, with the fully-connected layers encoding input-output associations. We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts. Experiments show our method outperforms standard attribution techniques in identifying hallucination causes and is supported by evidence from the model's training corpus. This work provides a unified perspective on hallucinations and a robust framework for their cause and analysis.
Equilibria in routing games with connected autonomous vehicles will not be strong, as exclusive clubs may form
Kucharski, Rafaล, Psarou, Anastasia, Descormier, Natello
User Equilibrium is the standard representation of the so-called routing game in which drivers adjust their route choices to arrive at their destinations as fast as possible. Asking whether this Equilibrium is strong or not was meaningless for human drivers who did not form coalitions due to technical and behavioral constraints. This is no longer the case for connected autonomous vehicles (CAVs), which will be able to communicate and collaborate to jointly form routing coalitions. We demonstrate this for the first time on a carefully designed toy-network example, where a `club` of three autonomous vehicles jointly decides to deviate from the user equilibrium and benefit (arrive faster). The formation of such a club has negative consequences for other users, who are not invited to join it and now travel longer, and for the system, making it suboptimal and disequilibrated, which triggers adaptation dynamics. This discovery has profound implications for the future of our cities. We demonstrate that, if not prevented, CAV operators may intentionally disequilibrate traffic systems from their classic Nash equilibria, benefiting their own users and imposing costs on others. These findings suggest the possible emergence of an exclusive CAV elite, from which human-driven vehicles and non-coalition members may be excluded, potentially leading to systematically longer travel times for those outside the coalition, which would be harmful for the equity of public road networks.
A Extensive-form correlated equilibrium
In the former, the mediator draws and recommends a complete normal-form plan to each player before the game starts. This is beneficial when the mediator wants to maximize, e.g., the Appendix A.1 provides a suitable formal definition of the set of EFCEs via the notion of trigger agent (originally This holds for arbitrary EFGs with multiple players and/or chance moves. Unfortunately, that algorithm is mainly a theoretical tool, and it is known to have limited scalability beyond toy problems. However, their algorithm is centralized and based on MCMC sampling which may limit its practical appeal. B.1 Proofs for Section 4 The following auxiliary result is exploited in the proof of Theorem 1. Lemma 4. This concludes the proof.Theorem 1.