Large language models (LLMs) are notorious for hallucinating, i.e., producing erroneous claims in their output. Such hallucinations can be dangerous, as occasional factual inaccuracies in the generated text might be obscured by the rest of the output being generally factually correct, making it extremely hard for the users to spot them. Current services that leverage LLMs usually do not provide any means for detecting unreliable generations. Here, we aim to bridge this gap. In particular, we propose a novel fact-checking and hallucination detection pipeline based on token-level uncertainty quantification. Uncertainty scores leverage information encapsulated in the output of a neural network or its layers to detect unreliable predictions, and we show that they can be used to fact-check the atomic claims in the LLM output. Moreover, we present a novel token-level uncertainty quantification method that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use. Our method Claim Conditioned Probability (CCP) measures only the uncertainty of a particular claim value expressed by the model. Experiments on the task of biography generation demonstrate strong improvements for CCP compared to the baselines for seven LLMs and four languages. Human evaluation reveals that the fact-checking pipeline based on uncertainty quantification is competitive with a fact-checking tool that leverages external knowledge.

The remarkable capability of large language models (LLMs) for in-context learning (ICL) needs to be activated by demonstration examples. Prior work has extensively explored the selection of examples for ICL, predominantly following the "select then organize" paradigm, such approaches often neglect the internal relationships between examples and exist an inconsistency between the training and inference. In this paper, we formulate the problem as a $Se$quential $Se$lection problem and introduce $Se^2$, a sequential-aware method that leverages the LLM's feedback on varying context, aiding in capturing inter-relationships and sequential information among examples, significantly enriching the contextuality and relevance of ICL prompts. Meanwhile, we utilize beam search to seek and construct example sequences, enhancing both quality and diversity. Extensive experiments across 23 NLP tasks from 8 distinct categories illustrate that $Se^2$ markedly surpasses competitive baselines and achieves 42\% relative improvement over random selection. Further in-depth analysis shows the effectiveness of proposed strategies, highlighting $Se^2$'s exceptional stability and adaptability across various scenarios. Code available at https://github.com/microsoft/LMOps.

This paper addresses the critical challenge of assessing the representativeness of news thumbnail images, which often serve as the first visual engagement for readers when an article is disseminated on social media. We focus on whether a news image represents the actors discussed in the news text. To serve the challenge, we introduce NewsTT, a manually annotated dataset of 1000 news thumbnail images and text pairs. We found that the pretrained vision and language models, such as BLIP-2, struggle with this task. Since news subjects frequently involve named entities or proper nouns, the pretrained models could have a limited capability to match news actors' visual and textual appearances. We hypothesize that learning to contrast news text with its counterfactual, of which named entities are replaced, can enhance the cross-modal matching ability of vision and language models. We propose CFT-CLIP, a contrastive learning framework that updates vision and language bi-encoders according to the hypothesis. We found that our simple method can boost the performance for assessing news thumbnail representativeness, supporting our assumption. Code and data can be accessed at https://github.com/ssu-humane/news-images-acl24.

Large language models (LLMs) demonstrate exceptional performance in numerous tasks but still heavily rely on knowledge stored in their parameters. Moreover, updating this knowledge incurs high training costs. Retrieval-augmented generation (RAG) methods address this issue by integrating external knowledge. The model can answer questions it couldn't previously by retrieving knowledge relevant to the query. This approach improves performance in certain scenarios for specific tasks. However, if irrelevant texts are retrieved, it may impair model performance. In this paper, we propose Retrieval Augmented Iterative Self-Feedback (RA-ISF), a framework that iteratively decomposes tasks and processes them in three submodules to enhance the model's problem-solving capabilities. Experiments show that our method outperforms existing benchmarks, performing well on models like GPT3.5, Llama2, significantly enhancing factual reasoning capabilities and reducing hallucinations.

Designed and manufactured in France, the Thalamus is a groundbreaking autonomous security robot that is redefining the way we think about surveillance and protection.

The world's first beauty pageant for AI women has finally revealed its shortlist of computer-generated ladies. Out of the 1,500 entrants into the Fanvue World AI Creator Awards, the 10 artificial finalist will battle it out to become the first ever Miss AI. And, with contestants ranging from the red-headed travel influencer Olivia C to Turkish model and astronaut Asena Ilik, competition is sure to be fierce. But, just like any other pageant, it will take more than good looks to win the prize and finalists will also be judged on their technology and social media clout. Fanvue co-founder Will Monange says: 'The awards have shown how engaged creators in the AI space are, and the standard of the shortlist is nothing short of incredible.'

Summer Games Fest kicks off this week, with its titular game showcase starting Friday. Expect a string of SGF events, livestreams, YouTube trailers to follow… and maybe a Silksong update. We'll be reporting live from LA, offering our thoughts and impressions of many of the games at the event -- especially if we get to play any of them. We break down every event right here, but the biggest events include the Summer Game Fest Live on Friday June 7 at 5PM ET, the Xbox Games Showcase on Sunday June 9, 1PM ET and Ubisoft Forward Monday June 10, 3PM ET. Wait, was there something else on that day?

Spotify is going through something right now. On Monday morning, the industry-defining audio streaming service announced that it would be hiking its Premium subscription prices for users in the United States, effective next month. The individual plan is rising by 1, the Duo plan by 2, and the family subscription by 3. These shifts arrive almost a year after Spotify raised U.S. subscription rates for the first time ever, upping the individual plan to 10.99 a month to match with competitors' price points. That increase was meant to mollify music-industry executives (who demanded better royalty payouts) and investors (who demanded that Spotify squeeze out regular profits).

Charles Yeh's battle with disinformation in Taiwan began with a bowl of beef noodles. Nine years ago, the Taiwanese engineer was at a restaurant with his family when his mother-in-law started picking the green onions out of her food. Asked what she was doing, she explained that onions can harm your liver. She knew this, she said, because she had received text messages telling her so. Yeh was puzzled by this. His family had always happily eaten green onions.

Trend filtering simplifies complex time series data by applying smoothness to filter out noise while emphasizing proximity to the original data. However, existing trend filtering methods fail to reflect abrupt changes in the trend due to `approximateness,' resulting in constant smoothness. This approximateness uniformly filters out the tail distribution of time series data, characterized by extreme values, including both abrupt changes and noise. In this paper, we propose Trend Point Detection formulated as a Markov Decision Process (MDP), a novel approach to identifying essential points that should be reflected in the trend, departing from approximations. We term these essential points as Dynamic Trend Points (DTPs) and extract trends by interpolating them. To identify DTPs, we utilize Reinforcement Learning (RL) within a discrete action space and a forecasting sum-of-squares loss function as a reward, referred to as the Dynamic Trend Filtering network (DTF-net). DTF-net integrates flexible noise filtering, preserving critical original subsequences while removing noise as required for other subsequences. We demonstrate that DTF-net excels at capturing abrupt changes compared to other trend filtering algorithms and enhances forecasting performance, as abrupt changes are predicted rather than smoothed out.