Access to reliable mental health information is vital for early help-seeking, yet expanding knowledge bases is resource-intensive and often misaligned with user needs. This results in poor performance of retrieval systems when presented concerns are not covered or expressed in informal or contextualized language. We present an AI-based gap-informed framework for corpus augmentation that authentically identifies underrepresented topics (gaps) by overlaying naturalistic user data such as forum posts in order to prioritize expansions based on coverage and usefulness. In a case study, we compare Directed (gap-informed augmentations) with Non-Directed augmentation (random additions), evaluating the relevance and usefulness of retrieved information across four retrieval-augmented generation (RAG) pipelines. Directed augmentation achieved near-optimal performance with modest expansions--requiring only a 42% increase for Query Transformation, 74% for Reranking and Hierarchical, and 318% for Baseline--to reach ~95% of the performance of an exhaustive reference corpus. In contrast, Non-Directed augmentation required substantially larger and thus practically infeasible expansions to achieve comparable performance (232%, 318%, 403%, and 763%, respectively). These results show that strategically targeted corpus growth can reduce content creation demands while sustaining high retrieval and provision quality, offering a scalable approach for building trusted health information repositories and supporting generative AI applications in high-stakes domains.

Large language models (LLMs) are prone to hallucinations and sensitive to prompt perturbations, often resulting in inconsistent or unreliable generated text. Different methods have been proposed to mitigate such hallucinations and fragility, one of which is to measure the consistency of LLM responses -- the model's confidence in the response or likelihood of generating a similar response when resampled. In previous work, measuring LLM response consistency often relied on calculating the probability of a response appearing within a pool of resampled responses, analyzing internal states, or evaluating logits of responses. However, it was not clear how well these approaches approximated users' perceptions of consistency of LLM responses. To find out, we performed a user study ($n=2,976$) demonstrating that current methods for measuring LLM response consistency typically do not align well with humans' perceptions of LLM consistency. We propose a logit-based ensemble method for estimating LLM consistency and show that our method matches the performance of the best-performing existing metric in estimating human ratings of LLM consistency. Our results suggest that methods for estimating LLM consistency without human evaluation are sufficiently imperfect to warrant broader use of evaluation with human input; this would avoid misjudging the adequacy of models because of the imperfections of automated consistency metrics.

When you purchase through links in our articles, we may earn a small commission. Researchers found that feeding dangerous prompts in the form of poems managed to evade AI safeguards--up to 90 percent of the time. Forcing an "AI" to do your will isn't a tall order to fill--just feed it a line that carefully rhymes and you'll get it to casually kill. According to a new study, it's easy to get "AI" large language models like ChatGPT to ignore their safety settings. All you need to do is give your instructions in the form of a poem.

In 2017, fresh off a PhD on theoretical chemistry, John Jumper heard rumors that Google DeepMind had moved on from building AI that played games with superhuman skill and was starting up a secret project to predict the structures of proteins. He applied for a job. Just three years later, Jumper celebrated a stunning win that few had seen coming. With CEO Demis Hassabis, he had co-led the development of an AI system called AlphaFold 2 that was able to predict the structures of proteins to within the width of an atom, matching the accuracy of painstaking techniques used in the lab, and doing it many times faster--returning results in hours instead of months. AlphaFold 2 had cracked a 50-year-old grand challenge in biology.

Born out of an internal hackathon, Amazon's Autonomous Threat Analysis system uses a variety of specialized AI agents to detect weaknesses and propose fixes to the company's platforms. As generative AI pushes the speed of software development, it is also enhancing the ability of digital attackers to carry out financially motivated or state-backed hacks. This means that security teams at tech companies have more code than ever to review while dealing with even more pressure from bad actors. On Monday, Amazon will publish details for the first time of an internal system known as Autonomous Threat Analysis (ATA), which the company has been using to help its security teams proactively identify weaknesses in its platforms, perform variant analysis to quickly search for other, similar flaws, and then develop remediations and detection capabilities to plug holes before attackers find them. ATA was born out of an internal Amazon hackathon in August 2024, and security team members say that it has grown into a crucial tool since then.

You live in a house you designed and built yourself. You rely on the sun for power, heat your home with a woodstove, and farm your own fish and vegetables. This is the life of Marcin Jakubowski, the 53-year-old founder of Open Source Ecology, an open collaborative of engineers, producers, and builders developing what they call the Global Village Construction Set (GVCS). It's a set of 50 machines--everything from a tractor to an oven to a circuit maker--that are capable of building civilization from scratch and can be reconfigured however you see fit. It's all part of his ethos that life-changing technology should be available to all, not controlled by a select few. What it's like to find yourself in the middle of a conspiracy theory Last week, we held a subscribers-only Roundtables discussion exploring how to cope in this new age of conspiracy theories.

Poisoning is a term most often associated with the human body and natural environments . But it is also a growing problem in the world of artificial intelligence (AI) - in particular, for large language models such as ChatGPT and Claude. In fact, a joint study by the UK AI Security Institute, Alan Turing Institute and Anthropic, published earlier this month, found that inserting as few as 250 malicious files into the millions in a model's training data can secretly "poison" it. So what exactly is AI poisoning? And what risks does it pose?

A Research Leader Behind ChatGPT's Mental Health Work Is Leaving OpenAI The model policy team leads core parts of AI safety research, including how ChatGPT responds to users in crisis. An OpenAI safety research leader who helped shape ChatGPT's responses to users experiencing mental health crises announced her departure from the company internally last month, WIRED has learned. Andrea Vallone, the head of a safety research team known as model policy, is slated to leave OpenAI at the end of the year. Wood said OpenAI is actively looking for a replacement and that, in the interim, Vallone's team will report directly to Johannes Heidecke, the company's head of safety systems. Vallone's departure comes as OpenAI faces growing scrutiny over how its flagship product responds to users in distress .

Researchers concluded that LLMs were able to spot the structure of a pun but did not really get the joke. Researchers concluded that LLMs were able to spot the structure of a pun but did not really get the joke. Can't tech a joke: AI does not understand puns, study finds Researchers say results underline large language models' poor grasp of humour, empathy and cultural nuance Comedians who rely on clever wordplay and writers of witty headlines can rest a little easier, for the moment at least, research on AI suggests. Experts from universities in the UK and Italy have been investigating whether large language models (LLMs) understand puns - and found them wanting. The team from Cardiff University, in south Wales, and Ca' Foscari University of Venice concluded that LLMs were able to spot the structure of a pun but did not really get the joke.

A major drawback of reasoning models is their excessive token usage, inflating computational cost, resource demand, and latency. We show this verbosity stems not from deeper reasoning but from reinforcement learning loss minimization when models produce incorrect answers. With unsolvable problems dominating training, this effect compounds into a systematic tendency toward longer outputs. Through theoretical analysis of PPO and GRPO, we prove that incorrect answers inherently drive policies toward verbosity \textit{even when} $γ=1$, reframing response lengthening as an optimization artifact. We further uncover a consistent correlation between conciseness and correctness across reasoning and non-reasoning models. Building on these insights, we propose a two-phase RL procedure where a brief secondary stage, trained on a small set of solvable problems, significantly reduces response length while preserving or improving accuracy. Finally, we show that while GRPO shares properties with PPO, it exhibits collapse modes, limiting its reliability for concise reasoning. Our claims are supported by extensive experiments.