We propose a novel cooperative herding strategy through backstepping control barrier functions (CBFs), which coordinates multiple herders to herd a group of evaders safely towards a designated goal region. For the herding system with heterogeneous groups involving herders and evaders, the behavior of the evaders can only be influenced indirectly by the herders' motion, especially when the evaders follow an inverse dynamics model and respond solely to repulsive interactions from the herders. This indirect interaction mechanism inherently renders the overall system underactuated. To address this issue, we first construct separate CBFs for the dual objectives of goal reaching and collision avoidance, which ensure both herding completion and safety guarantees. Then, we reformulate the underactuated herding dynamics into a control-affine structure and employ a backstepping approach to recursively design control inputs for the hierarchical barrier functions, avoiding taking derivatives of the higher-order system. Finally, we present a cooperative herding strategy based on backstepping CBFs that allow herders to safely herd multiple evaders into the goal region. In addition, centralized and decentralized implementations of the proposed algorithm are developed, further enhancing its flexibility and applicability. Extensive simulations and real-world experiments validate the effectiveness and safety of the proposed strategy in multi-robot herding.

-- We propose a decentralized reinforcement learning solution for multi-agent shepherding of non-cohesive targets using policy-gradient methods. This model-free framework effectively solves the shepherding problem without prior dynamics knowledge. Experiments demonstrate our method's effectiveness and scalability with increased target numbers and limited sensing capabilities. The shepherding problem in robotics exemplifies the problem of harnessing complex systems for control [1], [2]. It generally involves a group of actively controlled agents, termed herders, strategically influencing a group of passive agents, referred to as targets.

Herding-- one of humanity's most foundational ways of life--is a pillar of survival in West Africa's Sahel. Migratory herders usher cattle between seasonal pastures, since they rarely own land. However, these traditional ways of doing things are becoming increasingly impossible, thanks to a complex mix of climate change, politics and war. In more recent years, various Western players touting tech trends like artificial intelligence and predictive analysis have swooped in with promises to solve the region's myriad problems. But some think there could be a much simpler solution, that puts real data directly into the herders' hands. Recent advances in data collection--both from geosatellites and from herders themselves--have generated an abundance of information on ground cover quantity and quality, water availability, rain forecasts, livestock concentrations, and more.

This paper solves the problem of herding countless evaders by means of a few robots. The objective is to steer all the evaders towards a desired tracking reference while avoiding escapes. The problem is very challenging due to the highly complex repulsive evaders' dynamics and the underdetermined states to control. We propose a solution that is based on Implicit Control and a novel dynamic assignment strategy to select the evaders to be directly controlled. The former is a general technique that explicitly computes control inputs even in highly complex input-nonaffine dynamics. The latter is built upon a convex-hull dynamic clustering inspired by the Voronoi tessellation problem. The combination of both allows to choose the best evaders to directly control, while the others are indirectly controlled by exploiting the repulsive interactions among them. Simulations show that massive herds can be herd throughout complex patterns by means of a few herders.

In this Paper we propose a simple yet effective set of local control rules to make a group of "herder agents" collect and contain in a desired region an ensemble of non-cooperative stochastic "target agents" in the plane. We investigate the robustness of the proposed strategies to variations of the number of target agents and the strength of the repulsive force they feel when in proximity of the herders. Extensive numerical simulations confirm the effectiveness of the approach and are complemented by a more realistic validation on commercially available robotic agents via ROS.

We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms so as to improve the exploration of the state-space, and substantially reduce learning times by leveraging some limited knowledge of the plant encoded into a tutoring model-based control strategy. We illustrate the benefits of our novel approach and its effectiveness by using the problem of controlling one or more agents to herd and contain within a goal region a set of target free-roving agents in the plane.

EXTENDED ABSTRACT Model-free reinforcement learning (or simply reinforcement learning, RL, in what follows) is increasingly used in applications to solve a wide variety of control problems (Kober et al., 2013; Garcıa and Fern andez, 2015; Cheng et al., 2019). The lack of requiring a formal model of the plant renders it appealing for a heuristic, low-cost control design approach that can be easily implemented and adapted to different situations. As a tradeoff, learning processes often require a long training phase where the controller agent learns by trial-and-error how the plant responds to different control actions, and what actions to take to steer its behavior in a desired manner. This problem is particularly relevant when using tabular methods, such as Q-learning, in those situations where reinforcement learning is applied to control dynamical systems defined in continuous spaces (Lillicrap et al., 2019). It is therefore desirable to enhance the learning process by encoding some qualitative knowledge of the system dynamics via appropriate models.

Eat too much and there won't be grass for anyone. In an essay written in 1833, the British economist William Forster Lloyd made a profound observation using the example of cattle grazing. Lloyd described a hypothetical scenario involving herders who share a pasture, and individually decide how many of their animals would graze there. If few herders exercised restraint, overgrazing would occur, reducing the pasture's future usefulness and eventually hurting everybody. The sinister beauty of this example is that the rational course of action is to behave selfishly.