Personal
Introducing the NASA Onboard Artificial Intelligence Research (OnAIR) platform: an interview with Evana Gizzi
The Thirty-Seventh Annual Conference on Innovative Applications of Artificial Intelligence (IAAI 2025), which took place alongside AAAI 2025, serves as a showcase for successful applications and novel uses of AI. One such application is the Onboard Artificial Intelligence Research (OnAIR) platform, introduced by Evana Gizzi and colleagues in their paper OnAIR: Applications of The NASA On-Board Artificial Intelligence Research Platform. This open-source software pipeline and cognitive architecture tool has been designed to aid space research and missions. We spoke to Evana, Artificial Intelligence Research Lead at NASA Goddard Space Flight Center, about the OnAIR platform, some of the particular challenges of deploying AI-based solutions in space, and how the tool has been used so far. OnAIR is an open-source software pipeline and cognitive architecture tool.
ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving
Chen, Kai, Gao, Ruiyuan, Hong, Lanqing, Xu, Hang, Jia, Xu, Caesar, Holger, Dai, Dengxin, Liu, Bingbing, Tsishkou, Dzmitry, Xu, Songcen, Xu, Chunjing, Xu, Qiang, Lu, Huchuan, Yeung, Dit-Yan
In this paper, we present details of the 1st W-CODA workshop, held in conjunction with the ECCV 2024. W-CODA aims to explore next-generation solutions for autonomous driving corner cases, empowered by state-of-the-art multimodal perception and comprehension techniques. 5 Speakers from both academia and industry are invited to share their latest progress and opinions. We collect research papers and hold a dual-track challenge, including both corner case scene understanding and generation. As the pioneering effort, we will continuously bridge the gap between frontier autonomous driving techniques and fully intelligent, reliable self-driving agents robust towards corner cases.
Vision Transformer with Adversarial Indicator Token against Adversarial Attacks in Radio Signal Classifications
Zhang, Lu, Lambotharan, Sangarapillai, Zheng, Gan, Liao, Guisheng, Liu, Xuekang, Roli, Fabio, Maple, Carsten
--The remarkable success of transformers across various fields such as natural language processing and computer vision has paved the way for their applications in automatic modulation classification, a critical component in the communication systems of Internet of Things (IoT) devices. However, it has been observed that transformer-based classification of radio signals is susceptible to subtle yet sophisticated adversarial attacks. T o address this issue, we have developed a defensive strategy for transformer-based modulation classification systems to counter such adversarial attacks. In this paper, we propose a novel vision transformer (ViT) architecture by introducing a new concept known as adversarial indicator (AdvI) token to detect adversarial attacks. T o the best of our knowledge, this is the first work to propose an AdvI token in ViT to defend against adversarial attacks. Integrating an adversarial training method with a detection mechanism using AdvI token, we combine a training time defense and running time defense in a unified neural network model, which reduces architectural complexity of the system compared to detecting adversarial perturbations using separate models. We investigate into the operational principles of our method by examining the attention mechanism. We show the proposed AdvI token acts as a crucial element within the ViT, influencing attention weights and thereby highlighting regions or features in the input data that are potentially suspicious or anomalous. Through experimental results, we demonstrate that our approach surpasses several competitive methods in handling white-box attack scenarios, including those utilizing the fast gradient method, projected gradient descent attacks and basic iterative method. Lu Zhang is with School of Mathematics and Computer Science, Swansea university, Swansea, SA1 8EN, UK (e-mail: lu.zhang@swansea.ac.uk). Sangarapillai Lambotharan is with Institute for Digital Technologies, Loughborough University London, London, E20 3BS, UK (e-mail: s.lambotharan@lboro.ac.uk). Gan Zheng is with School of Engineering, University of Warwick, Coventry, CV4 7AL, UK (e-mail: gan.zheng@warwick.ac.uk). Guisheng Liao is with School of Electronic Engineering, Xidian University, Xi'an, 710071, People's Republic of China (e-mail: liaogs@xidian.edu.cn). Xuekang Liu is with the Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London, London, SW7 2AZ, U.K. (e-mail: xuekangliu@ieee.org).
AI companies start winning the copyright fight
If you need me after this newsletter publishes, I will be busy poring over photos from Jeff Bezos and Lauren Sanchez's wedding, the gaudiest and most star-studded affair to disrupt technology news this year. I found it a tacky and spectacular affair. Everyone who was anyone was there, except for Charlize Theron, who, unprompted, said on Monday: "I think we might be the only people who did not get an invite to the Bezos wedding. Judge William Alsup compared the Anthropic model's use of books to a "reader aspiring to be a writer." And the next day, Meta: The US district judge Vince Chhabria, in San Francisco, said in his decision on the Meta case that the authors had not presented enough evidence that the technology company's AI would cause "market dilution" by flooding the market with work similar to theirs. Judging by the rulings in favor of Meta and Anthropic, the authors are facing an uphill battle. Three weeks ago, Disney and NBCUniversal sued Midjourney, alleging that the ...
An interview with Nicolai Ommer: the RoboCupSoccer Small Size League
Kick-off in a Small Size League match. RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event is due to take place from 15-21 July in Salvador, Brazil. The Soccer component of RoboCup comprises a number of Leagues, with one of these being the Small Size League (SSL). We caught up with Executive Committee member Nicolai Ommer to find out more about the SSL, how the auto referees work, and how teams use AI.
Conversations with Andrea: Visitors' Opinions on Android Robots in a Museum
Heisler, Marcel, Becker-Asano, Christian
-- The android robot Andrea was set up at a public museum in Germany for six consecutive days to have conversations with visitors, fully autonomously. No specific context was given, so visitors could state their opinions regarding possible use-cases in structured interviews, without any bias. Additionally the 44 interviewees were asked for their general opinions of the robot, their reasons (not) to interact with it and necessary improvements for future use. The android's voice and wig were changed between different days of operation to give varying cues regarding its gender . This did not have a significant impact on the positive overall perception of the robot. Most visitors want the robot to provide information about exhibits in the future, while opinions on other roles, like a receptionist, were both wanted and explicitly not wanted by different visitors. Speaking more languages (than only English) and faster response times were the improvements most desired. These findings from the interviews are in line with an analysis of the system logs, which revealed, that after chitchat and personal questions, most of the 4436 collected requests asked for information related to the museum and to converse in a different language. The valuable insights gained from these real-world interactions are now used to improve the system to become a useful real-world application. An android robot's outer appearance is explicitly designed to resemble a human as closely as possible.
Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization
Zhang, Zhiwang, Xu, Dong, Ouyang, Wanli, Tan, Chuanqi
In this work, we propose a division-and-summarization (DaS) framework for dense video captioning. After partitioning each untrimmed long video as multiple event proposals, where each event proposal consists of a set of short video segments, we extract visual feature (e.g., C3D feature) from each segment and use the existing image/video captioning approach to generate one sentence description for this segment. Considering that the generated sentences contain rich semantic descriptions about the whole event proposal, we formulate the dense video captioning task as a visual cue aided sentence summarization problem and propose a new two stage Long Short Term Memory (LSTM) approach equipped with a new hierarchical attention mechanism to summarize all generated sentences as one descriptive sentence with the aid of visual features. Specifically, the first-stage LSTM network takes all semantic words from the generated sentences and the visual features from all segments within one event proposal as the input, and acts as the encoder to effectively summarize both semantic and visual information related to this event proposal. The second-stage LSTM network takes the output from the first-stage LSTM network and the visual features from all video segments within one event proposal as the input, and acts as the decoder to generate one descriptive sentence for this event proposal. Our comprehensive experiments on the ActivityNet Captions dataset demonstrate the effectiveness of our newly proposed DaS framework for dense video captioning.
RoboCupRescue: an interview with Adam Jacoff
RoboCup is an international scientific initiative with the goal of advancing the state of the science of intelligent robots, AI and automation. The annual RoboCup event will take place from 15-21 July in Salvador, Brazil. The RoboCupRescue League is an important element of the competition and focuses on the challenges involved in search and rescue applications. We caught up with Adam Jacoff, co-founder of the RoboCupRescue league, former RoboCup Trustee, and chair of the organising committee, to find out more. The RoboCupRescue League is now in its 25th year hosting competitions and workshops all around the world.
From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
Kim, Gyeongwon James, Wilf, Alex, Morency, Louis-Philippe, Fried, Daniel
Recent progress in autonomous code generation has fueled excitement around AI agents capable of accelerating scientific discovery by running experiments. However, there is currently no benchmark that evaluates whether such agents can implement scientific ideas when given varied amounts of code as a starting point, interpolating between reproduction (running code) and from-scratch replication (fully re-implementing and running code). We introduce AutoExperiment, a benchmark that evaluates AI agents' ability to implement and run machine learning experiments based on natural language descriptions in research papers. In each task, agents are given a research paper, a codebase with key functions masked out, and a command to run the experiment. The goal is to generate the missing code, execute the experiment in a sandboxed environment, and reproduce the results. AutoExperiment scales in difficulty by varying the number of missing functions $n$, ranging from partial reproduction to full replication. We evaluate state-of-the-art agents and find that performance degrades rapidly as $n$ increases. Agents that can dynamically interact with the environment (e.g. to debug their code) can outperform agents in fixed "agentless" harnesses, and there exists a significant gap between single-shot and multi-trial success rates (Pass@1 vs. Pass@5), motivating verifier approaches to our benchmark. Our findings highlight critical challenges in long-horizon code generation, context retrieval, and autonomous experiment execution, establishing AutoExperiment as a new benchmark for evaluating progress in AI-driven scientific experimentation. Our data and code are open-sourced at https://github.com/j1mk1m/AutoExperiment .
Making optimal decisions without having all the cards in hand
The article "Revelations: A Decidable Class of POMDP with Omega-Regular Objectives" won an Outstanding Paper Award at the AAAI 2025 conference, a prestigious international conference about artificial intelligence. This year, only three papers received such an award out of 3,000 accepted and 12,000 submitted! This recognition crowns the results of research initiated in Bordeaux (France) within the Synthèse team at the Bordeaux Computer Science Research Laboratory (LaBRI), where four of the authors work: Marius Belly, Nathanaël Fijalkow, Hugo Gimbert, and Pierre Vandenhove. The work also involved researchers from Paris (Florian Horn) and Antwerp (Guillermo A. Pérez). The article is freely available on arXiv, and this post outlines its main ideas.