Goto

Collaborating Authors

 history


Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models

Neural Information Processing Systems

Clinical reasoning in medicine is a hypothesis-driven process where physicians refine diagnoses from limited information through targeted history, physical examination, and diagnostic investigations. In contrast, current medical benchmarks for large language models (LLMs) primarily assess knowledge recall through single-turn questions, where complete clinical information is provided upfront. To address this gap, we introduce VivaBench, a multi-turn benchmark that evaluates sequential clinical reasoning in LLM agents. Our dataset comprises 1152 physiciancurated clinical vignettes structured as interactive scenarios that simulate a viva voce examination in medical training, requiring agents to actively probe for relevant findings, select appropriate investigations, and synthesize information across multiple steps to reach a diagnosis. We evaluated several state-of-the-art LLMs and found that while models demonstrate competence in diagnosing conditions within well-described clinical presentations, their performance degrades significantly when required to navigate diagnostic uncertainty. Our analysis identified several failure modes that mirror common issues in clinical practice, including: (1) fixation on initial hypotheses, (2) excessive investigation ordering, (3) premature diagnostic closure, and (4) missing critical conditions. These patterns reveal fundamental limitations in how current LLMs manage uncertainty and gather information sequentially. Through VivaBench, we provide a standardized benchmark for evaluating conversational medical AI systems for real-world clinical decision support. Beyond medical applications, we contribute to the larger corpus of research on agentic AI by demonstrating how sequential reasoning trajectories can diverge in complex decision-making environments.


Why only humans sleepwalk

Popular Science

It's a trait evolution forgot to get rid of. More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. A colorized film still from the 1931 German film'Emil and the Detectives' shows a man sleepwalking. Breakthroughs, discoveries, and DIY tips sent six days a week. By signing up, you confirm you are 16+, will receive newsletters and promotional content and agree to our Terms of Use and acknowledge the data practices in our Privacy Policy .


In 1962 Wisconsin, delivery pizzas were cooked in traffic

Popular Science

Mobile kitchens ensured that pizzas arrived piping hot. More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. In 1962, Pizza on Wheels aimed to deliver restaurant-fresh pizza straight from the oven. Breakthroughs, discoveries, and DIY tips sent six days a week. By signing up, you confirm you are 16+, will receive newsletters and promotional content and agree to our Terms of Use and acknowledge the data practices in our Privacy Policy .


Evaluating LLMs in Open-Source Games

Neural Information Processing Systems

Large Language Models' (LLMs) programming capabilities enable their participation in open-source games: a game-theoretic setting in which players submit computer programs in lieu of actions. These programs offer numerous advantages, including interpretability, inter-agent transparency, and formal verifiability; additionally, they enable program equilibria, solutions that leverage the transparency of code and are inaccessible within normal-form settings. We evaluate the capabilities of leading open-and closed-weight LLMs to predict and classify program strategies and evaluate features of the approximate program equilibria reached by LLM agents in dyadic and evolutionary settings. We identify the emergence of payoffmaximizing, cooperative, and deceptive strategies, characterize the adaptation of mechanisms within these programs over repeated open-source games, and analyze their comparative evolutionary fitness. We find that open-source games serve as a viable environment to study and steer the emergence of cooperative strategy in multi-agent dilemmas.


Scalable Policy-Based RLAlgorithms for POMDPs

Neural Information Processing Systems

The continuous nature of belief states in POMDPs presents significant computational challenges in learning the optimal policy. In this paper, we consider an approach that solves a Partially Observable Reinforcement Learning (PORL) problem by approximating the corresponding POMDP model into a finite-state Markov Decision Process (MDP) (called Superstate MDP). We first derive theoretical guarantees that improve upon prior work that relate the optimal value function of the transformed Superstate MDP to the optimal value function of the original POMDP. Next, we propose a policy-based learning approach with linear function approximation to learn the optimal policy for the Superstate MDP. Consequently, our approach shows that a POMDP can be approximately solved using TD-learning followed by Policy Optimization by treating it as an MDP, where the MDP state corresponds to a finite history. We show that the approximation error decreases exponentially with the length of this history. To the best of our knowledge, our finite-time bounds are the first to explicitly quantify the error introduced when applying standard TD learning to a setting where the true dynamics are not Markovian.


Locked Out of the World Cup: A Year Marked by Barriers, Borders, and Broken Access

WIRED

The 2026 World Cup promises a global celebration. Many Arab fans may find themselves excluded. For the first time in World Cup history, eight Arab nations have qualified for this year's tournament, including Morocco, Tunisia, Egypt, Algeria, Saudi Arabia, Qatar, Iraq, and Jordan--double the number of teams that qualified for Qatar in 2022. Yet, the tournament is taking place at an unprecedented moment of heightened geopolitical tension. The US-Israel war with Iran, which began in February of this year, has caused ripple effects across Gulf states and neighboring countries in the Levant, including Lebanon, Palestine, and Jordan, reshaping the security around travel and mobility for fans and players hailing from the region. The US State Department has fully suspended visa issuance for nationals from countries with teams that qualified, including Iran and Haiti--despite it being the first time Haiti has qualified for a World Cup since 1974.


Diffusion Guided Adversarial State Perturbations in Reinforcement Learning

Neural Information Processing Systems

Reinforcement learning (RL) systems, while achieving remarkable success across various domains, are vulnerable to adversarial attacks. This is especially a concern in vision-based environments where minor manipulations of high-dimensional image inputs can easily mislead the agent's behavior. To this end, various defenses have been proposed recently, with state-of-the-art approaches achieving robust performance even under large state perturbations. However, after closer investigation, we found that the effectiveness of the current defenses is due to a fundamental weakness of the existing lp norm-constrained attacks, which can barely alter the semantics of image input even under a relatively large perturbation budget. In this work, we propose SHIFT, a novel policy-agnostic diffusion-based state perturbation attack to go beyond this limitation. Our attack is able to generate perturbed states that are semantically different from the true states while remaining realistic and history-aligned to avoid detection. Evaluations show that our attack effectively breaks existing defenses, including the most sophisticated ones, significantly outperforming existing attacks while being more perceptually stealthy.


Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models

Neural Information Processing Systems

We present a neural network structure, FramePack, to train next-frame (or nextframe-section) prediction models for video generation. FramePack compresses input frame contexts with frame-wise importance so that more frames can be encoded within a fixed context length, with more important frames having longer contexts. The frame importance can be measured using time proximity, feature similarity, or hybrid metrics. The packing method allows for inference with thousands of frames and training with relatively large batch sizes. We also present drift prevention methods to address observation bias (error accumulation), including early-established endpoints, adjusted sampling orders, and discrete history representation.


SpaceX IPO raised 10bn more than thought

BBC News

SpaceX raised $10bn (£7.5bn) more than initially thought when it sold shares to the public on Friday - bringing in a total of $85.7bn. Elon Musk's rocket and Artificial Intellgience (AI) company pulled off the biggest initial public offering (IPO) in history when it joined New York's Nasdaq stock exchange last week. The listing had raised $75bn from investors, which Musk told employees will be spent funding a significant growth phase. But the banks which backed the IPO exercised a so-called greenshoe clause, which let them purchase an extra $10bn of SpaceX shares. The extra $10bn raised, revealed in a statement by SpaceX announcing the completion of the listing, would by itself rank as one of the biggest IPOs in history.


Realistic Doctor-Patient Interactions

Neural Information Processing Systems

Doctor-patient consultations require multi-turn, context-aware communication tailored to diverse patient personas. Training or evaluating doctor LLMs in such settings requires realistic patient interaction systems. However, existing simulators often fail to reflect the full range of personas seen in clinical practice. To address this, we introduce PATIENTSIM, a patient simulator that generates realistic and diverse patient personas for clinical scenarios, grounded in medical expertise. PATIENTSIM operates using: 1) clinical profiles, including symptoms and medical history, derived from real-world data in the MIMIC-ED and MIMIC-IV datasets, and 2) personas defined by four axes: personality, language proficiency, medical history recall level, and cognitive confusion level, resulting in 37 unique combinations. We evaluate eight LLMs for factual accuracy and persona consistency. The top-performing open-source model, Llama 3.3 70B, is validated by four clinicians to confirm the robustness of our framework. As an open-source, customizable platform, PATIENTSIM provides a reproducible and scalable solution that can be customized for specific training needs. Offering a privacy-compliant environment, it serves as a robust testbed for evaluating medical dialogue systems across diverse patient presentations and shows promise as an educational tool for healthcare.