The adaptivity and maneuvering capabilities of Autonomous Underwater Vehicles (AUVs) have drawn significant attention in oceanic research, due to the unpredictable disturbances and strong coupling among the AUV's degrees of freedom. In this paper, we developed large language model (LLM)-enhanced reinforcement learning (RL)-based adaptive S-surface controller for AUVs. Specifically, LLMs are introduced for the joint optimization of controller parameters and reward functions in RL training. Using multi-modal and structured explicit task feedback, LLMs enable joint adjustments, balance multiple objectives, and enhance task-oriented performance and adaptability. In the proposed controller, the RL policy focuses on upper-level tasks, outputting task-oriented high-level commands that the S-surface controller then converts into control signals, ensuring cancellation of nonlinear effects and unpredictable external disturbances in extreme sea conditions. Under extreme sea conditions involving complex terrain, waves, and currents, the proposed controller demonstrates superior performance and adaptability in high-level tasks such as underwater target tracking and data collection, outperforming traditional PID and SMC controllers.

This study presents an Actor-Critic Cooperative Compensated Model Predictive Controller (AC3MPC) designed to address unknown system dynamics. To avoid the difficulty of modeling highly complex dynamics and ensuring realtime control feasibility and performance, this work uses deep reinforcement learning with a model predictive controller in a cooperative framework to handle unknown dynamics. The model-based controller takes on the primary role as both controllers are provided with predictive information about the other. This improves tracking performance and retention of inherent robustness of the model predictive controller. We evaluate this framework for off-road autonomous driving on unknown deformable terrains that represent sandy deformable soil, sandy and rocky soil, and cohesive clay-like deformable soil. Our findings demonstrate that our controller statistically outperforms standalone model-based and learning-based controllers by upto 29.2% and 10.2%. This framework generalized well over varied and previously unseen terrain characteristics to track longitudinal reference speeds with lower errors. Furthermore, this required significantly less training data compared to purely learning-based controller, while delivering better performance even when under-trained.

Designing ground-aerial robots is challenging due to the increased actuation requirements which can lead to added weight and reduced locomotion efficiency. Morphobots mitigate this by combining actuators into multi-functional groups and leveraging ground transformation to achieve different locomotion modes. However, transforming on the ground requires dealing with the complexity of ground-vehicle interactions during morphing, limiting applicability on rough terrain. Mid-air transformation offers a solution to this issue but demands operating near or beyond actuator limits while managing complex aerodynamic forces. We address this problem by introducing the Aerially Transforming Morphobot (ATMO), a robot which transforms near the ground achieving smooth transition between aerial and ground modes. To achieve this, we leverage the near ground aerodynamics, uncovered by experimental load cell testing, and stabilize the system using a model-predictive controller that adapts to ground proximity and body shape. The system is validated through numerous experimental demonstrations. We find that ATMO can land smoothly at body postures past its actuator saturation limits by virtue of the uncovered ground-effect.

Algorithmic predictions are increasingly informing societal resource allocations by identifying individuals for targeting. Policymakers often build these systems with the assumption that by gathering more observations on individuals, they can improve predictive accuracy and, consequently, allocation efficiency. An overlooked yet consequential aspect of prediction-driven allocations is that of timing. The planner has to trade off relying on earlier and potentially noisier predictions to intervene before individuals experience undesirable outcomes, or they may wait to gather more observations to make more precise allocations. We examine this tension using a simple mathematical model, where the planner collects observations on individuals to improve predictions over time. We analyze both the ranking induced by these predictions and optimal resource allocation. We show that though individual prediction accuracy improves over time, counter-intuitively, the average ranking loss can worsen. As a result, the planner's ability to improve social welfare can decline. We identify inequality as a driving factor behind this phenomenon. Our findings provide a nuanced perspective and challenge the conventional wisdom that it is preferable to wait for more accurate predictions to ensure the most efficient allocations.

In reinforcement learning (RL), world models serve as internal simulators, enabling agents to predict environment dynamics and future outcomes in order to make informed decisions. While previous approaches leveraging discrete latent spaces, such as DreamerV3, have demonstrated strong performance in discrete action settings and visual control tasks, their comparative performance in state-based continuous control remains underexplored. In contrast, methods with continuous latent spaces, such as TD-MPC2, have shown notable success in state-based continuous control benchmarks. In this paper, we demonstrate that modeling discrete latent states has benefits over continuous latent states and that discrete codebook encodings are more effective representations for continuous control, compared to alternative encodings, such as one-hot and label-based encodings. Based on these insights, we introduce DCWM: Discrete Codebook World Model, a self-supervised world model with a discrete and stochastic latent space, where latent states are codes from a codebook. We combine DCWM with decision-time planning to get our model-based RL algorithm, named DC-MPC: Discrete Codebook Model Predictive Control, which performs competitively against recent state-of-the-art algorithms, including TD-MPC2 and DreamerV3, on continuous control benchmarks. See our project website www.aidanscannell.com/dcmpc.

ExAMPC: the Data-Driven Explainable and Approximate NMPC with Physical Insights Jean Pierre Allamaa 1, 2 and Panagiotis Patrinos 2 and Tong Duy Son 1 Abstract -- Amidst the surge in the use of Artificial Intelligence (AI) for control purposes, classical and model-based control methods maintain their popularity due to their transparency and deterministic nature. However, advanced controllers like Nonlinear Model Predictive Control (NMPC), despite proven capabilities, face adoption challenges due to their computational complexity and unpredictable closed-loop performance in complex validation systems. This paper introduces ExAMPC, a methodology bridging classical control and explainable AI by augmenting the NMPC with data-driven insights to improve the trustworthiness and reveal the optimization solution and closed-loop performance's sensitivities to physical variables and system parameters. By employing a low-order spline embedding to reduce the open-loop trajectory dimensionality by over 95%, and integrating it with SHAP and Symbolic Regression from eXplainable AI (XAI) for an approximate NMPC, we enable intuitive physical insights into the NMPC's optimization routine. The prediction accuracy of the approximate NMPC is enhanced through physics-inspired continuous-time constraints penalties, reducing the predicted continuous trajectory violations by 93%. ExAMPC enables accurate forecasting of the NMPC's computational requirements with explainable insights on worst-case scenarios. Experimental validation on automated valet parking and autonomous racing with lap-time optimization NMPC, demonstrates the methodology's practical effectiveness in real-world applications. I. INTRODUCTION Linear Model Predictive Control (MPC) stands out for its inherent explainability, allowing precise analysis of the instantaneous open-loop (OL) prediction and closed-loop (CL) system behavior. However, this clarity on stability and performance diminishes with complex systems, such as chaotic dynamics or those involving a plant model that is more complicated than the linear prediction model in the MPC.

Full-waveform inversion (FWI) is an advanced technique for reconstructing high-resolution subsurface physical parameters by progressively minimizing the discrepancy between observed and predicted seismic data. However, conventional FWI encounters challenges in real data applications, primarily due to its conventional objective of direct measurements of the data misfit. Accurate estimation of the source wavelet is essential for effective data fitting, alongside the need for low-frequency data and a reasonable initial model to prevent cycle skipping. Additionally, wave equation solvers often struggle to accurately simulate the amplitude of observed data in real applications. To address these challenges, we introduce a correlation-based source-independent objective function for FWI that aims to mitigate source uncertainty and amplitude dependency, which effectively enhances its practicality for real data applications. We develop a deep-learning framework constrained by this new objective function with a velocity-distribution supported deep image prior, which reparameterizes velocity inversion into trainable parameters within an autoencoder, thereby reducing the nonlinearity in the conventional FWI's objective function. We demonstrate the superiority of our proposed method using synthetic data from benchmark velocity models and, more importantly, two real datasets. These examples highlight its effectiveness and practicality even under challenging conditions, such as missing low frequencies, a crude initial velocity model, and an incorrect source wavelet.

The rapid evolution of machine learning (ML) has brought about groundbreaking developments in numerous industries, not the least of which is in the area of undersea communication. This domain is critical for applications like ocean exploration, environmental monitoring, resource management, and national security. Bangladesh, a maritime nation with abundant resources in the Bay of Bengal, can harness the immense potential of ML to tackle the unprecedented challenges associated with underwater communication. Beyond that, environmental conditions are unique to the region: in addition to signal attenuation, multipath propagation, noise interference, and limited bandwidth. In this study, we address the necessity to bring ML into communication via undersea; it investigates the latest technologies under the domain of ML in that respect, such as deep learning and reinforcement learning, especially concentrating on Bangladesh scenarios in the sense of implementation. This paper offers a contextualized regional perspective by incorporating region-specific needs, case studies, and recent research to propose a roadmap for deploying ML-driven solutions to improve safety at sea, promote sustainable resource use, and enhance disaster response systems. This research ultimately highlights the promise of ML-powered solutions for transforming undersea communication, leading to more efficient and cost-effective technologies that subsequently contribute to both economic growth and environmental sustainability.

In software engineering chatrooms, communication is often hindered by imprecise questions that cannot be answered. Recognizing key entities can be essential for improving question clarity and facilitating better exchange. However, existing research using natural language processing techniques often overlooks these software-specific nuances. In this paper, we introduce Software-specific Named Entity Recognition, Intent Detection, and Resolution Classification (SENIR), a labeling approach that leverages a Large Language Model to annotate entities, intents, and resolution status in developer chatroom conversations. To offer quantitative guidance for improving question clarity and resolvability, we build a resolution prediction model that leverages SENIR's entity and intent labels along with additional predictive features. We evaluate SENIR on the DISCO dataset using a subset of annotated chatroom dialogues. SENIR achieves an 86% F-score for entity recognition, a 71% F-score for intent detection, and an 89% F-score for resolution status classification. Furthermore, our resolution prediction model, tested with various sampling strategies (random undersampling and oversampling with SMOTE) and evaluation methods (5-fold cross-validation, 10-fold cross-validation, and bootstrapping), demonstrates AUC values ranging from 0.7 to 0.8. Key factors influencing resolution include positive sentiment and entities such as Programming Language and User Variable across multiple intents, while diagnostic entities are more relevant in error-related questions. Moreover, resolution rates vary significantly by intent: questions about API Usage and API Change achieve higher resolution rates, whereas Discrepancy and Review have lower resolution rates. A Chi-Square analysis confirms the statistical significance of these differences.

Existing contextual multi-armed bandit (MAB) algorithms fail to effectively capture both long-term trends and local patterns across all arms, leading to suboptimal performance in environments with rapidly changing reward structures. They also rely on static exploration rates, which do not dynamically adjust to changing conditions. To overcome these limitations, we propose LNUCB-TA, a hybrid bandit model integrating a novel nonlinear component (adaptive k-Nearest Neighbors (k-NN)) for reducing time complexity, alongside a global-and-local attention-based exploration mechanism. Our approach uniquely combines linear and nonlinear estimation techniques, with the nonlinear module dynamically adjusting k based on reward variance to enhance spatiotemporal pattern recognition. This reduces the likelihood of selecting suboptimal arms while improving reward estimation accuracy and computational efficiency. The attention-based mechanism ranks arms by past performance and selection frequency, dynamically adjusting exploration and exploitation in real time without requiring manual tuning of exploration rates. By integrating global attention (assessing all arms collectively) and local attention (focusing on individual arms), LNUCB-TA efficiently adapts to temporal and spatial complexities. Empirical results show LNUCB-TA significantly outperforms state-of-the-art linear, nonlinear, and hybrid bandits in cumulative and mean reward, convergence, and robustness across different exploration rates. Theoretical analysis further confirms its reliability with a sub-linear regret bound.