Global Convergence of Direct Policy Search for State-Feedback \mathcal{H}_\infty Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferential

Direct policy search has been widely applied in modern reinforcement learning and continuous control. However, the theoretical properties of direct policy search on nonsmooth robust control synthesis have not been fully understood. The optimal \mathcal{H}_\infty control framework aims at designing a policy to minimize the closed-loop \mathcal{H}_\infty norm, and is arguably the most fundamental robust control paradigm. In this work, we show that direct policy search is guaranteed to find the global solution of the robust \mathcal{H}_\infty state-feedback control design problem. Notice that policy search for optimal \mathcal{H}_\infty control leads to a constrained nonconvex nonsmooth optimization problem, where the nonconvex feasible set consists of all the policies stabilizing the closed-loop dynamics.