issue
Fact-checking with Generative AI: A Systematic Cross-Topic Examination of LLMs Capacity to Detect Veracity of Political Information
Kuznetsova, Elizaveta, Vitulano, Ilaria, Makhortykh, Mykola, Stolze, Martha, Nagy, Tomas, Vziatysheva, Victoria
The purpose of this study is to assess how large language models (LLMs) can be used for fact-checking and contribute to the broader debate on the use of automated means for veracity identification. To achieve this purpose, we use AI auditing methodology that systematically evaluates performance of five LLMs (ChatGPT 4, Llama 3 (70B), Llama 3.1 (405B), Claude 3.5 Sonnet, and Google Gemini) using prompts regarding a large set of statements fact-checked by professional journalists (16,513). Specifically, we use topic modeling and regression analysis to investigate which factors (e.g. topic of the prompt or the LLM type) affect evaluations of true, false, and mixed statements. Our findings reveal that while ChatGPT 4 and Google Gemini achieved higher accuracy than other models, overall performance across models remains modest. Notably, the results indicate that models are better at identifying false statements, especially on sensitive topics such as COVID-19, American political controversies, and social issues, suggesting possible guardrails that may enhance accuracy on these topics. The major implication of our findings is that there are significant challenges for using LLMs for factchecking, including significant variation in performance across different LLMs and unequal quality of outputs for specific topics which can be attributed to deficits of training data. Our research highlights the potential and limitations of LLMs in political fact-checking, suggesting potential avenues for further improvements in guardrails as well as fine-tuning.
- North America > United States (1.00)
- Asia > Russia (0.46)
- Europe > Switzerland (0.14)
- Europe > Germany (0.14)
- Media > News (1.00)
- Health & Medicine (1.00)
- Government > Regional Government > North America Government > United States Government (0.95)
Robust and Conjugate Spatio-Temporal Gaussian Processes
Laplante, William, Altamirano, Matias, Duncan, Andrew, Knoblauch, Jeremias, Briol, François-Xavier
State-space formulations allow for Gaussian process (GP) regression with linear-in-time computational cost in spatio-temporal settings, but performance typically suffers in the presence of outliers. In this paper, we adapt and specialise the robust and conjugate GP (RCGP) framework of Altamirano et al. (2024) to the spatio-temporal setting. In doing so, we obtain an outlier-robust spatio-temporal GP with a computational cost comparable to classical spatio-temporal GPs. We also overcome the three main drawbacks of RCGPs: their unreliable performance when the prior mean is chosen poorly, their lack of reliable uncertainty quantification, and the need to carefully select a hyperparameter by hand. We study our method extensively in finance and weather forecasting applications, demonstrating that it provides a reliable approach to spatio-temporal modelling in the presence of outliers.
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.14)
- Europe > United Kingdom > England > Greater London > London (0.04)
- North America > United States > Virginia > Arlington County > Arlington (0.04)
- (2 more...)
Essex dog attack: Man arrested after woman dies in Jaywick
A woman has died after being attacked by two dogs in Essex. She was found seriously injured in the village of Jaywick, near Clacton-on-Sea, at about 16:00 GMT on Saturday and a 39-year-old man has been arrested. Officers said the woman died at the scene and the man was being held on suspicion of dangerous dog offences. Two dogs have been destroyed and the investigation into the circumstances of the woman's death is continuing, Essex Police said. It added the breed of the dogs involved had not yet been confirmed and urged people not to speculate.Image source, Ed SchoolingImage caption, Crowds watched as a helicopter took off from the Jaywick Sands beach Ch Supt Glen Pavelin said: "My thoughts, and those of our officers and staff, are with the family of the woman who died yesterday. "This incident will be a huge shock to the community and I understand their concerns.
- Europe > United Kingdom > England (0.41)
- North America > United States (0.16)
- North America > El Salvador (0.16)
- (10 more...)
Five Minutes of AI - Issue #109
Welcome to your official daily AI learning resource and news for AI students, researchers, professionals, or even enthusiasts! I share a resource that will teach you something in a few minutes daily, along with exciting news that is worth your time. Keep working on your AI skills, learning new things, and staying up-to-date in a few minutes daily! "Learn what is machine learning (ML), how it works, and its importance in five minutes". Researchers have come up with a new and improved technology called SuperGPS that is accurate within 10 centimeters (or 3.9 inches) and doesn't rely on navigation satellite systems.
- Government > Space Agency (0.40)
- Government > Regional Government > North America Government > United States Government (0.40)
AI Disruption: Issue #3
I'm bringing you all some of the AI-related news and developments I found most interesting over the past week. Read on to learn more about neural networks, a cooperative AI infrastructure, AI crypto trading bots, and AI/human visual processing. A team of researchers at Los Alamos National Laboratory has developed a novel approach for comparing neural networks. According to the team, this new approach looks within the "black box" of artificial intelligence, and it helps them understand neural network behavior. A team of scientists at Incheon National University in South Korea has designed a cooperative infrastructure for artificial intelligence (AI)-assisted aerial and ground operations by using UAVs and mobile robots.
- North America > United States > New Mexico > Los Alamos County > Los Alamos (0.27)
- Asia > South Korea > Incheon > Incheon (0.27)
Issue #18 - Friday September 23, 2022
What's New @ NeuML publishes interesting content covering our open source projects, services and insights. The scheduled frequency is weekly to monthly. Work is underway for txtai 5.0. Here is an illustration that further explains graph path traversal. The chart below shows GitHub star growth to date with annotations.
AI Research Codes are Open, Accessibility is an Issue
For the past few years, the scientific community worldwide has been advocating the accessibility of science. 'Open Science', as they call it, is an ongoing movement to make research papers accessible to all. Open information is vital for research, even in space tech. Not many know that three years ago, when scientists created the first-ever black hole image, it was made possible only because of an open-source software, Matplotlib. The research papers that often claim to have their dataset/code open, are often found to be making false proclamations.
- Media (0.33)
- Health & Medicine > Nuclear Medicine (0.31)