research paper
Google's AI can turn almost anything you read into a podcast
The Dell Laptop Buying Guide 2026: Which XPS or Alienware Laptop Should You Buy? The Dell Laptop Buying Guide UK: Which Dell Laptop Should You Buy? When you purchase through links in our articles, we may earn a small commission. Google's AI can turn almost anything you read into a podcast Google's Gemini Notebook and Illuminate can turn long documents and dense research papers into easy-to-follow and engaging audio discussions between two AI hosts. Part of my job here at PCWorld is taking complex topics and breaking them down into easy-to-understand, approachable content.
A bust-up between mathematicians is fuelling fears that AI could undermine science
Just two boxes may hold 10 times more information than all the text on the internet, says Dr Oliver Vince, the facility's co-founder. His firm, Basecamp Research, plans to collect as much DNA as possible to create an artificial intelligence that can speak the language of biology, in the same way that US and Chinese tech firms have hoovered up written material online to create Large Language Models (LLMs), the basis of current AI systems. Vince, like many in business and academic research, believes AI will supercharge scientific research. But increasingly, it also comes with controversy. Last month, OpenAI, the creator of ChatGPT, said it had solved one of maths' great unsolved problems, relating to the famous Navier-Stokes equations.
ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Code
Large language models (LLMs) have shown promise in transforming machine learning research, yet their capability to faithfully implement genuinely novel ideas from recent research papers--ideas unseen during pretraining--remains unclear. We introduce ResearchCodeBench, a benchmark that evaluates LLMs' ability to translate cutting-edge ML contributions from top 2024-2025 research papers into executable code. We assessed 30+ proprietary and open-source LLMs, finding that even the best models correctly implement less than 40% of the code.
ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers?
Ye, Christine, Yuan, Sihan, Cooray, Suchetha, Dillmann, Steven, Roque, Ian L. V., Baron, Dalya, Frank, Philipp, Martin-Alvarez, Sergio, Koblischke, Nolan, Qu, Frank J, Yang, Diyi, Wechsler, Risa, Ciuca, Ioana
Frontier AI agents show increasing promise as scientific research assistants, and may eventually be useful for extended, open-ended research workflows. However, in order to use agents for novel research, we must first assess the underlying faithfulness and correctness of their work. To evaluate agents as research assistants, we introduce ReplicationBench, an evaluation framework that tests whether agents can replicate entire research papers drawn from the astrophysics literature. Astrophysics, where research relies heavily on archival data and computational study while requiring little real-world experimentation, is a particularly useful testbed for AI agents in scientific research. We split each paper into tasks which require agents to replicate the paper's core contributions, including the experimental setup, derivations, data analysis, and codebase. Each task is co-developed with the original paper authors and targets a key scientific result, enabling objective evaluation of both faithfulness (adherence to original methods) and correctness (technical accuracy of results). ReplicationBench is extremely challenging for current frontier language models: even the best-performing language models score under 20%. We analyze ReplicationBench trajectories in collaboration with domain experts and find a rich, diverse set of failure modes for agents in scientific research. ReplicationBench establishes the first benchmark of paper-scale, expert-validated astrophysics research tasks, reveals insights about agent performance generalizable to other domains of data-driven science, and provides a scalable framework for measuring AI agents' reliability in scientific research.
A Position Paper on the Automatic Generation of Machine Learning Leaderboards
Timmer, Roelien C, Hou, Yufang, Wan, Stephen
An important task in machine learning (ML) research is comparing prior work, which is often performed via ML leaderboards: a tabular overview of experiments with comparable conditions (e.g., same task, dataset, and metric). However, the growing volume of literature creates challenges in creating and maintaining these leaderboards. To ease this burden, researchers have developed methods to extract leaderboard entries from research papers for automated leaderboard curation. Yet, prior work varies in problem framing, complicating comparisons and limiting real-world applicability. In this position paper, we present the first overview of Automatic Leaderboard Generation (ALG) research, identifying fundamental differences in assumptions, scope, and output formats. We propose an ALG unified conceptual framework to standardise how the ALG task is defined. We offer ALG benchmarking guidelines, including recommendations for datasets and metrics that promote fair, reproducible evaluation. Lastly, we outline challenges and new directions for ALG, such as, advocating for broader coverage by including all reported results and richer metadata.
Jr. AI Scientist and Its Risk Report: Autonomous Scientific Exploration from a Baseline Paper
Miyai, Atsuyuki, Toyooka, Mashiro, Otonari, Takashi, Zhao, Zaiying, Aizawa, Kiyoharu
Understanding the current capabilities and risks of AI Scientist systems is essential for ensuring trustworthy and sustainable AI-driven scientific progress while preserving the integrity of the academic ecosystem. To this end, we develop Jr. AI Scientist, a state-of-the-art autonomous AI scientist system that mimics the core research workflow of a novice student researcher: Given the baseline paper from the human mentor, it analyzes its limitations, formulates novel hypotheses for improvement, and iteratively conducts experiments until improvements are realized, and writes a paper with the results. Unlike previous approaches that assume full automation or operate on small-scale code, Jr. AI Scientist follows a well-defined research workflow and leverages modern coding agents to handle complex, multi-file implementations, leading to scientifically valuable contributions. Through our experiments, the Jr. AI Scientist successfully generated new research papers that build upon real NeurIPS, IJCV, and ICLR works by proposing and implementing novel methods. For evaluation, we conducted automated assessments using AI Reviewers, author-led evaluations, and submissions to Agents4Science, a venue dedicated to AI-driven scientific contributions. The findings demonstrate that Jr. AI Scientist generates papers receiving higher review scores than existing fully automated systems. Nevertheless, we identify important limitations from both the author evaluation and the Agents4Science reviews, indicating the potential risks of directly applying current AI Scientist systems and key challenges for future research. Finally, we comprehensively report various risks identified during development. We believe this study clarifies the current role and limitations of AI Scientist systems, offering insights into the areas that still require human expertise and the risks that may emerge as these systems evolve.
The Denario project: Deep knowledge AI agents for scientific discovery
Villaescusa-Navarro, Francisco, Bolliet, Boris, Villanueva-Domingo, Pablo, Bayer, Adrian E., Acquah, Aidan, Amancharla, Chetana, Barzilay-Siegal, Almog, Bermejo, Pablo, Bilodeau, Camille, Ramírez, Pablo Cárdenas, Cranmer, Miles, França, Urbano L., Hahn, ChangHoon, Jiang, Yan-Fei, Jimenez, Raul, Lee, Jun-Young, Lerario, Antonio, Mamun, Osman, Meier, Thomas, Ojha, Anupam A., Protopapas, Pavlos, Roy, Shimanto, Spergel, David N., Tarancón-Álvarez, Pedro, Tiwari, Ujjwal, Viel, Matteo, Wadekar, Digvijay, Wang, Chi, Wang, Bonny Y., Xu, Licong, Yovel, Yossi, Yue, Shuwen, Zhou, Wen-Han, Zhu, Qiyao, Zou, Jiajun, Zubeldia, Íñigo
We present Denario, an AI multi-agent system designed to serve as a scientific research assistant. Denario can perform many different tasks, such as generating ideas, checking the literature, developing research plans, writing and executing code, making plots, and drafting and reviewing a scientific paper. The system has a modular architecture, allowing it to handle specific tasks, such as generating an idea, or carrying out end-to-end scientific analysis using Cmbagent as a deep-research backend. In this work, we describe in detail Denario and its modules, and illustrate its capabilities by presenting multiple AI-generated papers generated by it in many different scientific disciplines such as astrophysics, biology, biophysics, biomedical informatics, chemistry, material science, mathematical physics, medicine, neuroscience and planetary science. Denario also excels at combining ideas from different disciplines, and we illustrate this by showing a paper that applies methods from quantum physics and machine learning to astrophysical data. We report the evaluations performed on these papers by domain experts, who provided both numerical scores and review-like feedback. We then highlight the strengths, weaknesses, and limitations of the current system. Finally, we discuss the ethical implications of AI-driven research and reflect on how such technology relates to the philosophy of science. We publicly release the code at https://github.com/AstroPilot-AI/Denario. A Denario demo can also be run directly on the web at https://huggingface.co/spaces/astropilot-ai/Denario, and the full app will be deployed on the cloud.
GAMER PAT: Research as a Serious Game
As generative AI increasingly outperforms students in producing academic writing, a critical question arises: how can we preserve the motivation, creativity, and intellectual growth of novice researchers in an age of automated academic achievement? This paper introduces GAMER PAT (GAme MastER, Paper Authoring Tutor), a prompt-engineered AI chatbot that reframes research paper writing as a serious game. Through role-playing mechanics, users interact with a co-author NPC and anonymous reviewer NPCs, turning feedback into "missions" and advancing through a narrative-driven writing process. Our study reports on 26+ gameplay chat logs, including both autoethnography and use by graduate students under supervision. Using qualitative log analysis with SCAT (Steps for Coding and Theorization), we identified an emergent four-phase scaffolding pattern: (1) question posing, (2) meta-perspective, (3) structuring, and (4) recursive reflection. These results suggest that GAMER PAT supports not only the structural development of research writing but also reflective and motivational aspects. We present this work as a descriptive account of concept and process, not a causal evaluation. We also include a speculative outlook envisioning how humans may continue to cultivate curiosity and agency alongside AI-driven research. This arXiv version thus provides both a descriptive report of design and usage, and a forward-looking provocation for future empirical studies.
Human-Agent Collaborative Paper-to-Page Crafting for Under $0.1
Ma, Qianli, Wang, Siyu, Chen, Yilin, Tang, Yinhao, Yang, Yixiang, Guo, Chang, Gao, Bingjie, Xing, Zhening, Sun, Yanan, Zhang, Zhipeng
In the quest for scientific progress, communicating research is as vital as the discovery itself. Yet, researchers are often sidetracked by the manual, repetitive chore of building project webpages to make their dense papers accessible. While automation has tackled static slides and posters, the dynamic, interactive nature of webpages has remained an unaddressed challenge. To bridge this gap, we reframe the problem, arguing that the solution lies not in a single command, but in a collaborative, hierarchical process. We introduce $\textbf{AutoPage}$, a novel multi-agent system that embodies this philosophy. AutoPage deconstructs paper-to-page creation into a coarse-to-fine pipeline from narrative planning to multimodal content generation and interactive rendering. To combat AI hallucination, dedicated "Checker" agents verify each step against the source paper, while optional human checkpoints ensure the final product aligns perfectly with the author's vision, transforming the system from a mere tool into a powerful collaborative assistant. To rigorously validate our approach, we also construct $\textbf{PageBench}$, the first benchmark for this new task. Experiments show AutoPage not only generates high-quality, visually appealing pages but does so with remarkable efficiency in under 15 minutes for less than \$0.1. Code and dataset will be released at $\href{https://mqleet.github.io/AutoPage_ProjectPage/}{Webpage}$.