XX 1 Disturbance Compensation for Safe Kinematic Control of Robotic Systems with Closed Architecture Fan Zhang 1,2, Jinfeng Chen 1, Joseph J. B. Mvogo Ahanda 3, Hanz Richter 4, Ge Lv 5, Bin Hu 1,2, Qin Lin 1,2 Abstract--In commercial robotic systems, it is common to encounter a closed inner-loop (low-level) torque controller that is not user-modifiable. However, the outer-loop controller, which sends kinematic commands such as position or velocity for the inner-loop controller to track, is typically exposed to users. In this work, we focus on the development of an easily integrated add-on at the outer-loop layer by combining disturbance rejection control and robust control barrier function for high-performance tracking and safe control of the whole dynamic system of an industrial manipulator . This is particularly beneficial when 1) the inner-loop controller is imperfect, unmodifiable, and uncertain; and 2) the dynamic model exhibits significant uncertainty. Stability analysis, formal safety guarantee proof, simulations, and hardware experiments with a PUMA robotic manipulator are presented. Our solution demonstrates superior performance in terms of simplicity of implementation, robustness, tracking precision, and safety compared to the state of the art. I. INTRODUCTION Robotic systems often employ hierarchical software design, stacking perception, decision-making, planning, and low-level control. Such modularity is particularly beneficial for troubleshooting and improving the reliability of robotic systems. For example, in the control block, a combination of a kinematic controller (outer-loop controller) and a dynamic controller (inner-loop controller) is commonly seen in various robots. However, because tuning the inner-loop controller requires expert knowledge, this component is typically not exposed to users due to product safety considerations, a practice referred to as closed architecture in the literature [1]-[4]. In other words, users are only allowed to design the kinematic controller, sending position or velocity for the inner-loop controller to track. Additionally, mechanical parts 1 The authors are with the Department of Engineering Technology, University of Houston, USA. Corresponding author: Qin Lin, qlin21@central.uh.edu 2 Fan Zhang is also with the Department of Electrical and Computer Engineering, University of Houston, USA 3 Joseph Jean Baptiste Mvogo Ahanda is with the Department of Biomedical Engineering, The University of Ebolowa, Cameroon 4 Hanz Richter is with the Department of Mechanical Engineering, Cleveland State University, USA 5 Ge Lv is with the Department of Mechanical Engineering, Clemson University, USA. This material is based upon work supported by the National Science Foundation under Grant Nos.

We address the problem of safely learning controlled stochastic dynamics from discrete-time trajectory observations, ensuring system trajectories remain within predefined safe regions during both training and deployment. Safety-critical constraints of this kind are crucial in applications such as autonomous robotics, finance, and biomedicine. We introduce a method that ensures safe exploration and efficient estimation of system dynamics by iteratively expanding an initial known safe control set using kernel-based confidence bounds. After training, the learned model enables predictions of the system's dynamics and permits safety verification of any given control. Our approach requires only mild smoothness assumptions and access to an initial safe control set, enabling broad applicability to complex real-world systems. We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics. Experimental evaluations demonstrate the practical effectiveness of our method in terms of safety, estimation accuracy, and computational efficiency.

Reinforcement Learning (RL) is a powerful method for controlling dynamic systems, but its learning mechanism can lead to unpredictable actions that undermine the safety of critical systems. Here, we propose RL with Adaptive Regularization (RL-AR), an algorithm that enables safe RL exploration by combining the RL policy with a policy regularizer that hard-codes the safety constraints. RL-AR performs policy combination via a "focus module," which determines the appropriate combination depending on the state--relying more on the safe policy regularizer for less-exploited states while allowing unbiased convergence for well-exploited states. In a series of critical control applications, we demonstrate that RL-AR not only ensures safety during training but also achieves a return competitive with the standards of model-free RL that disregards safety.

-- In this work, we explore the application of barrier states (BaS) in the realm of safe nonlinear adaptive control. Our proposed framework derives barrier states for systems with parametric uncertainty, which are augmented into the uncertain dynamical model. We employ an adaptive nonlinear control strategy based on a control Lyapunov functions approach to design a stabilizing controller for the augmented system. The developed theory shows that the controller ensures safe control actions for the original system while meeting specified performance objectives. We validate the effectiveness of our approach through simulations on diverse systems, including a planar quadrotor subject to unknown drag forces and an adaptive cruise control system, for which we provide comparisons with existing methodologies. Safe control methods have increasingly gained attention in recent research due to their importance in ensuring system reliability. Many of these methods rely on the notion of set invariance and detailed system models to maintain safety.

An interesting theoretical bound on the error of the posterior mean of GPR by bounding Lipschitz constants. Unlike previous work such bound is easier to evaluate and therefore it can be more practical, as it has been shown by the application to safe RL. Given the importance of GPs to sequential decision making under uncertainty the paper will be of interest to many practitioners. Please note that this submission caused a huge amount of discussion around conference policy issues regarding slicing contributions: "Note that slicing contributions too thinly may result in submissions being deemed dual submissions. Specifically, a case of slicing too thinly may correspond to two submissions by the same authors that are so similar that publishing one would render the other too incremental to be accepted."

In this paper, we develop a novel closed-form Control Barrier Function (CBF) and associated controller shield for the Kinematic Bicycle Model (KBM) with respect to obstacle avoidance. The proposed CBF and shield -- designed by an algorithm we call ShieldNN -- provide two crucial advantages over existing methodologies. First, ShieldNN considers steering and velocity constraints directly with the non-affine KBM dynamics; this is in contrast to more general methods, which typically consider only affine dynamics and do not guarantee invariance properties under control constraints. Second, ShieldNN provides a closed-form set of safe controls for each state unlike more general methods, which typically rely on optimization algorithms to generate a single instantaneous for each state. Together, these advantages make ShieldNN uniquely suited as an efficient Multi-Obstacle Safe Actions (i.e. multiple-barrier-function shielding) during training time of a Reinforcement Learning (RL) enabled NN controller. We show via experiments that ShieldNN dramatically increases the completion rate of RL training episodes in the presence of multiple obstacles, thus establishing the value of ShieldNN in training RL-based controllers.

Data-driven models are subject to model errors due to limited and noisy training data. Key to the application of such models in safety-critical domains is the quantification of their model error. Gaussian processes provide such a measure and uniform error bounds have been derived, which allow safe control based on these models. However, existing error bounds require restrictive assumptions. In this paper, we employ the Gaussian process distribution and continuity arguments to derive a novel uniform error bound under weaker assumptions.

Autonomous systems have witnessed a rapid increase in their capabilities, but it remains a challenge for them to perform tasks both effectively and safely. The fact that performance and safety can sometimes be competing objectives renders the cooptimization between them difficult. One school of thought is to treat this cooptimization as a constrained optimal control problem with a performance-oriented objective function and safety as a constraint. However, solving this constrained optimal control problem for general nonlinear systems remains challenging. In this work, we use the general framework of constrained optimal control, but given the safety state constraint, we convert it into an equivalent control constraint, resulting in a state and time-dependent control-constrained optimal control problem. This equivalent optimal control problem can readily be solved using the dynamic programming principle. We show the corresponding value function is a viscosity solution of a certain Hamilton-Jacobi-Bellman Partial Differential Equation (HJB-PDE). Furthermore, we demonstrate the effectiveness of our method with a two-dimensional case study, and the experiment shows that the controller synthesized using our method consistently outperforms the baselines, both in safety and performance.

Deep reinforcement learning (DRL) has demonstrated impressive performance in many continuous control tasks. However, one major stumbling block to the real-world application of DRL is the lack of safety guarantees. Although DRL agents can statisfy the system safety in expectation through reward shaping, it is quite challenging to design the DRL agent to consistently meet hard constraints (e.g., safety specification) at every time step. On the other hand, existing works in the field of safe control provide guarantees on the persistent satisfaction of hard safety constraints. However, the explicit analytical system dynamics models are required in order to synthesize the safe control, and the dynamics models are typically not accessible in DRL settings. In this paper, we present a model-free safe control algorithm, implicit safe set algorithm, for synthesizing safeguards for DRL agents that will assure provable safety throughout training. The proposed algorithm synthesizes a safety index (also called the barrier certificate) and a subsequent safe control law only by querying a black-box dynamic function (e.g., a digital twin simulator). Moreover, we theoretically prove that the implicit safe set algorithm guarantees finite time convergence to the safe set and forward invariance for both continuous-time and discrete-time systems. We validate the proposed implicit safe set algorithm on the state-of-the-art safety benchmark Safety Gym, where the proposed method achieves zero safety violations and gains 95% 9% cumulative reward compared to state-of-the-art safe DRL methods.

Generating safe behaviors for autonomous systems is important as they continue to be deployed in the real world, especially around people. In this work, we focus on developing a novel safe controller for systems where there are multiple sources of uncertainty. We formulate a novel multimodal safe control method, called the Multimodal Safe Set Algorithm (MMSSA) for the case where the agent has uncertainty over which discrete mode the system is in, and each mode itself contains additional uncertainty. To our knowledge, this is the first energy-function-based safe control method applied to systems with multimodal uncertainty. We apply our controller to a simulated human-robot interaction where the robot is uncertain of the human's true intention and each potential intention has its own additional uncertainty associated with it, since the human is not a perfectly rational actor. We compare our proposed safe controller to existing safe control methods and find that it does not impede the system performance (i.e. efficiency) while also improving the safety of the system.