Energy
AerOSeg: Harnessing SAM for Open-Vocabulary Segmentation in Remote Sensing Images
Dutta, Saikat, Vasim, Akhil, Gole, Siddhant, Rezatofighi, Hamid, Banerjee, Biplab
Image segmentation beyond predefined categories is a key challenge in remote sensing, where novel and unseen classes often emerge during inference. Open-vocabulary image Segmentation addresses these generalization issues in traditional supervised segmentation models while reducing reliance on extensive per-pixel annotations, which are both expensive and labor-intensive to obtain. Most Open-Vocabulary Segmentation (OVS) methods are designed for natural images but struggle with remote sensing data due to scale variations, orientation changes, and complex scene compositions. This necessitates the development of OVS approaches specifically tailored for remote sensing. In this context, we propose AerOSeg, a novel OVS approach for remote sensing data. First, we compute robust image-text correlation features using multiple rotated versions of the input image and domain-specific prompts. These features are then refined through spatial and class refinement blocks. Inspired by the success of the Segment Anything Model (SAM) in diverse domains, we leverage SAM features to guide the spatial refinement of correlation features. Additionally, we introduce a semantic back-projection module and loss to ensure the seamless propagation of SAM's semantic information throughout the segmentation pipeline. Finally, we enhance the refined correlation features using a multi-scale attention-aware decoder to produce the final segmentation map. We validate our SAM-guided Open-Vocabulary Remote Sensing Segmentation model on three benchmark remote sensing datasets: iSAID, DLRSD, and OpenEarthMap. Our model outperforms state-of-the-art open-vocabulary segmentation methods, achieving an average improvement of 2.54 h-mIoU.
Compliant Explicit Reference Governor for Contact Friendly Robotic Manipulators
Gautam, Yaashia, Nechyporenko, Nataliya, Lin, Chi-Hui, Roncone, Alessandro, Nicotra, Marco M.
-- This paper introduces the Compliant Explicit Reference Governor (C-ERG), an extension of the Explicit Reference Governor that allows the robot to operate safely while in contact with the environment. The C-ERG is an intermediate layer that can be placed between a high-level planner and a low-level controller: its role is to enforce operational constraints and to enable the smooth transition between free-motion and contact operations. The C-ERG ensures safety by limiting the total energy available to the robotic arm at the time of contact. In the absence of contact, however, the C-ERG does not penalize the system performance. The emerging trend in the modern industry is to prioritize flexibility [1]. Until recently, robotics has been dominated by sampling-based motion planning which places an emphasis on "collision-free" paths to avoid harming itself or anything in its path [2].
Deploying Large AI Models on Resource-Limited Devices with Split Federated Learning
Qiang, Xianke, Liu, Hongda, Zhang, Xinran, Chang, Zheng, Liang, Ying-Chang
Abstract--Large Artificial Intelligence Models (LAMs) powered by massive datasets, extensive parameter scales, and extensive computational resources, leading to significant transformations across various industries. Y et, their practical deployment on resource-limited mobile edge devices is hindered by critical challenges such as data privacy, constrained resources, and high overhead costs. Addressing this gap, this paper proposes a novel framework, named Quantized Split Federated Fine-T uning Large AI Model (SFLAM). By partitioning the training load between edge devices and servers using a split learning paradigm, SFLAM can facilitate the operation of large models on devices and significantly lowers the memory requirements on edge devices. Additionally, SFLAM incorporates quantization management, power control, and bandwidth allocation strategies to enhance training efficiency while concurrently reducing energy consumption and communication latency. A theoretical analysis exploring the latency-energy trade-off is presented, and the framework's efficacy is validated via comprehensive simulations. The findings indicate that SFLAM achieves superior performance in terms of learning efficiency and scalability compared to conventional methods, thereby providing a valuable approach for enabling advanced AI services in resource-constrained scenarios. I. Introduction The advent of Large AI Models (LAMs), such as Chat-GPT and DeepSeek, marked a significant leap in AI capabilities, powered by their extensive parameter scales, large-scale datasets, and substantial computational resources [1]. As user demand for ubiquitous AI access and real-time, personalized experiences grows, deploying and training these models on mobile devices becomes increasingly relevant [2]. T o meet these escalating demands, fine-tuning, which involves adapting pre-trained models with domain-specific data, has become a widely adopted and efficient strategy for enhancing LAM performance on specialized tasks, offering a cost-effective path to superior results.
InterQ: A DQN Framework for Optimal Intermittent Control
Aggarwal, Shubham, Maity, Dipankar, Baลar, Tamer
In this letter, we explore the communication-control co-design of discrete-time stochastic linear systems through reinforcement learning. Specifically, we examine a closed-loop system involving two sequential decision-makers: a scheduler and a controller. The scheduler continuously monitors the system's state but transmits it to the controller intermittently to balance the communication cost and control performance. The controller, in turn, determines the control input based on the intermittently received information. Given the partially nested information structure, we show that the optimal control policy follows a certainty-equivalence form. Subsequently, we analyze the qualitative behavior of the scheduling policy. To develop the optimal scheduling policy, we propose InterQ, a deep reinforcement learning algorithm which uses a deep neural network to approximate the Q-function. Through extensive numerical evaluations, we analyze the scheduling landscape and further compare our approach against two baseline strategies: (a) a multi-period periodic scheduling policy, and (b) an event-triggered policy. The results demonstrate that our proposed method outperforms both baselines. The open source implementation can be found at https://github.com/AC-sh/InterQ.
A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7
Lee, Hojoon, Seno, Takuma, Tai, Jun Jet, Subramanian, Kaushik, Kawamoto, Kenta, Stone, Peter, Wurman, Peter R.
Deep reinforcement learning has achieved superhuman racing performance in high-fidelity simulators like Gran Turismo 7 (GT7). It typically utilizes global features that require instrumentation external to a car, such as precise localization of agents and opponents, limiting real-world applicability. To address this limitation, we introduce a vision-based autonomous racing agent that relies solely on ego-centric camera views and onboard sensor data, eliminating the need for precise localization during inference. This agent employs an asymmetric actor-critic framework: the actor uses a recurrent neural network with the sensor data local to the car to retain track layouts and opponent positions, while the critic accesses the global features during training. Evaluated in GT7, our agent consistently outperforms GT7's built-drivers. To our knowledge, this work presents the first vision-based autonomous racing agent to demonstrate champion-level performance in competitive racing scenarios.
CL-CoTNav: Closed-Loop Hierarchical Chain-of-Thought for Zero-Shot Object-Goal Navigation with Vision-Language Models
Cai, Yuxin, He, Xiangkun, Wang, Maonan, Guo, Hongliang, Yau, Wei-Yun, Lv, Chen
Visual Object Goal Navigation (ObjectNav) requires a robot to locate a target object in an unseen environment using egocentric observations. However, decision-making policies often struggle to transfer to unseen environments and novel target objects, which is the core generalization problem. Traditional end-to-end learning methods exacerbate this issue, as they rely on memorizing spatial patterns rather than employing structured reasoning, limiting their ability to generalize effectively. In this letter, we introduce Closed-Loop Hierarchical Chain-of-Thought Navigation (CL-CoTNav), a vision-language model (VLM)-driven ObjectNav framework that integrates structured reasoning and closed-loop feedback into navigation decision-making. To enhance generalization, we fine-tune a VLM using multi-turn question-answering (QA) data derived from human demonstration trajectories. This structured dataset enables hierarchical Chain-of-Thought (H-CoT) prompting, systematically extracting compositional knowledge to refine perception and decision-making, inspired by the human cognitive process of locating a target object through iterative reasoning steps. Additionally, we propose a Closed-Loop H-CoT mechanism that incorporates detection and reasoning confidence scores into training. This adaptive weighting strategy guides the model to prioritize high-confidence data pairs, mitigating the impact of noisy inputs and enhancing robustness against hallucinated or incorrect reasoning. Extensive experiments in the AI Habitat environment demonstrate CL-CoTNav's superior generalization to unseen scenes and novel object categories. Our method consistently outperforms state-of-the-art approaches in navigation success rate (SR) and success weighted by path length (SPL) by 22.4\%. We release our datasets, models, and supplementary videos on our project page.
Bidirectional Linear Recurrent Models for Sequence-Level Multisource Fusion
Liu, Qisai, Jiang, Zhanhong, Waite, Joshua R., Liu, Chao, Balu, Aditya, Sarkar, Soumik
Sequence modeling is a critical yet challenging task with wide-ranging applications, especially in time series forecasting for domains like weather prediction, temperature monitoring, and energy load forecasting. Transformers, with their attention mechanism, have emerged as state-of-the-art due to their efficient parallel training, but they suffer from quadratic time complexity, limiting their scalability for long sequences. In contrast, recurrent neural networks (RNNs) offer linear time complexity, spurring renewed interest in linear RNNs for more computationally efficient sequence modeling. In this work, we introduce BLUR (Bidirectional Linear Unit for Recurrent network), which uses forward and backward linear recurrent units (LRUs) to capture both past and future dependencies with high computational efficiency. BLUR maintains the linear time complexity of traditional RNNs, while enabling fast parallel training through LRUs. Furthermore, it offers provably stable training and strong approximation capabilities, making it highly effective for modeling long-term dependencies. Extensive experiments on sequential image and time series datasets reveal that BLUR not only surpasses transformers and traditional RNNs in accuracy but also significantly reduces computational costs, making it particularly suitable for real-world forecasting tasks. Our code is available here.
Combining Forecasts using Meta-Learning: A Comparative Study for Complex Seasonality
Abstract--In this paper, we investigate meta-learning for combining forecasts generated by models of different types . While typical approaches for combining forecasts involve s imple averaging, machine learning techniques enable more sophis ti-cated methods of combining through meta-learning, leading to improved forecasting accuracy. We use linear regression, k - nearest neighbors, multilayer perceptron, random forest, and long short-term memory as meta-learners. We define global and local meta-learning variants for time series with compl ex seasonality and compare meta-learners on multiple forecas ting problems, demonstrating their superior performance compa red to simple averaging. Ensemble methods are widely recognized as a cornerstone of modern machine learning (ML) [1], commonly used for regression and classification problems. In addition, ensem bling has proven to be a highly effective approach for increasing the predictive power of forecasting models. The ensemble approach in forecasting, which involves combining the predictions of multiple models, can be justified for several reasons. First of all, it usually leads to increased accurac y. Ensemble models often outperform individual models, as the y leverage the strengths of different models and minimize the ir weaknesses. By combining diverse models, the ensemble can produce more accurate predictions by capturing a broader range of patterns and insights from the data. Ensembling als o allows for the incorporation of multiple drivers into the da ta generating process, mitigating uncertainties regarding m odel form and parameter specification [2].
Hybrid AI-Physical Modeling for Penetration Bias Correction in X-band InSAR DEMs: A Greenland Case Study
Mansour, Islam, Fischer, Georg, Haensch, Ronny, Hajnsek, Irena
Digital elevation models derived from Interferometric Synthetic Aperture Radar (InSAR) data over glacial and snow-covered regions often exhibit systematic elevation errors, commonly termed "penetration bias. " W e leverage existing physics-based models and propose an integrated correction framework that combines parametric physical modeling with machine learning. W e evaluate the approach across three distinct training scenarios -- each defined by a different set of acquisition parameters -- to assess overall performance and the model's ability to generalize. Our experiments on Greenland's ice sheet using T anDEM-X data show that the proposed hybrid model corrections significantly reduce the mean and standard deviation of DEM errors compared to a purely physical modeling baseline. The hybrid framework also achieves significantly improved generalization than a pure ML approach when trained on data with limited diversity in acquisition parameters.
A temporal scale transformer framework for precise remaining useful life prediction in fuel cells
Tang, Zezhi, Chen, Xiaoyu, Jin, Xin, Zhang, Benyuan, Liang, Wenyu
In exploring Predictive Health Management (PHM) strategies for Proton Exchange Membrane Fuel Cells (PEMFC), the Transformer model, widely used in data-driven approaches, excels in many fields but struggles with time series analysis due to its self-attention mechanism, which yields a complexity of the input sequence squared and low computational efficiency. It also faces challenges in capturing both global long-term dependencies and local details effectively. To tackle this, we propose the Temporal Scale Transformer (TSTransformer), an enhanced version of the inverted Transformer (iTransformer). Unlike traditional Transformers that treat each timestep as an input token, TSTransformer maps sequences of varying lengths into tokens at different stages for inter-sequence modeling, using attention to capture multivariate correlations and feed-forward networks (FFN) to encode sequence representations. By integrating a one-dimensional convolutional layer into the multivariate attention for multi-level scaling of K and V matrices, it improves local feature extraction, captures temporal scale characteristics, and reduces token count and computational costs. Experiments comparing TSTransformer with models like Long Short-Term Memory, iTransformer, and Transformer demonstrate its potential as a powerful tool for advancing PHM in renewable energy, effectively addressing the limitations of pure Transformer models in data-driven time series tasks.