This paper proposes a comprehensive hierarchical control framework for autonomous decision-making arising in robotics and autonomous systems. In a typical hierarchical control architecture, high-level decision making is often characterised by discrete state and decision/control sets. However, a rational decision is usually affected by not only the discrete states of the autonomous system, but also the underlying continuous dynamics even the evolution of its operational environment. This paper proposes a holistic and comprehensive design process and framework for this type of challenging problems, from new modelling and design problem formulation to control design and stability analysis. It addresses the intricate interplay between traditional continuous systems dynamics utilized at the low levels for control design and discrete Markov decision processes (MDP) for facilitating high-level decision making. We model the decision making system in complex environments as a hybrid system consisting of a controlled MDP and autonomous (i.e. uncontrolled) continuous dynamics. Consequently, the new formulation is called as hybrid Markov decision process (HMDP). The design problem is formulated with a focus on ensuring both safety and optimality while taking into account the influence of both the discrete and continuous state variables of different levels. With the help of the model predictive control (MPC) concept, a decision maker design scheme is proposed for the proposed hybrid decision making model. By carefully designing key ingredients involved in this scheme, it is shown that the recursive feasibility and stability of the proposed autonomous decision making scheme are guaranteed. The proposed framework is applied to develop an autonomous lane changing system for intelligent vehicles.

This paper proposes a new 3D gas distribution mapping technique based on the local message passing of Gaussian belief propagation that is capable of resolving in real time, concentration estimates in 3D space whilst accounting for the obstacle information within the scenario, the first of its kind in the literature. The gas mapping problem is formulated as a 3D factor graph of Gaussian potentials, the connections of which are conditioned on local occupancy values. The Gaussian belief propagation framework is introduced as the solver and a new hybrid message scheduler is introduced to increase the rate of convergence. The factor graph problem is then redesigned as a dynamically expanding inference task, coupling the information of consecutive gas measurements with local spatial structure obtained by the robot. The proposed algorithm is compared to the state of the art methods in 2D and 3D simulations and is found to resolve distribution maps orders of magnitude quicker than typical direct solvers. The proposed framework is then deployed for the first time onboard a ground robot in a 3D mapping and exploration task. The system is shown to be able to resolve multiple sensor inputs and output high resolution 3D gas distribution maps in a GPS denied cluttered scenario in real time. This online inference of complicated plume structures provides a new layer of contextual information over its 2D counterparts and enables autonomous systems to take advantage of real time estimates to inform potential next best sampling locations.

This paper presents stability analysis tools for model predictive control (MPC) with and without terminal weight. Stability analysis of MPC with a limited horizon but without terminal weight is a long-standing open problem. By using a modified value function as an Lyapunov function candidate and the principle of optimality, this paper establishes stability conditions for this type of widely spread MPC algorithms. A new stability guaranteed MPC algorithm without terminal weight (MPCS) is presented. With the help of designing a new sublevel set defined by the value function of one-step ahead stage cost, conditions for checking its recursive feasibility and stability of the proposed MPC algorithm are presented. The new stability condition and the derived MPCS overcome the difficulties arising in the existing terminal weight based MPC framework, including the need of searching a suitable terminal weight and possible poor performance caused by an inappropriate terminal weight. This work is further extended to MPC with a terminal weight for the completeness. Numerical examples are presented to demonstrate the effectiveness of the proposed tool, whereas the existing stability analysis tools are either not applicable or lead to quite conservative results. It shows that the proposed tools offer a number of mechanisms to achieve stability: adjusting state and/or control weights, extending the length of horizon, and adding a simple extra constraint on the first or second state in the optimisation.