Remarks on the Polyak-Lojasiewicz inequality and the convergence of gradient systems

-- This work explores generalizations of the Polyak- Łojasiewicz inequality (PŁI) and their implications for the convergence behavior of gradient flows in optimization problems. Motivated by the continuous-time linear quadratic regulator (CT -LQR) policy optimization problem - where only a weaker version of the PŁI is characterized in the literature - this work shows that while weaker conditions are sufficient for global convergence to, and optimality of the set of critical points of the cost function, the "profile" of the gradient flow solution can change significantly depending on which "flavor" of inequality the cost satisfies. After a general theoretical analysis, we focus on fitting the CT -LQR policy optimization problem to the proposed framework, showing that, in fact, it can never satisfy a PŁI in its strongest form. Recent advances in Artificial Intelligence (AI) and Machine Learning (ML) have rekindled interest in optimization theory, with many traditional results being revisited in light of the proposed techniques [1]-[8]. In particular, the typical model-free formulation of many successful learning techniques motivates the study of gradient-based optimization methods, which are invaluable in understanding the training of neural networks and similar architectures, typically done through back-propagation algorithms. Gradient descent or, in continuous time, gradient flow, consists in searching for the argument x that minimizes the value of a given function f (x) by "moving along" the direction of steepest descent of the cost function. Theoretical guarantees are typically desirable, and in search of balancing generality and good properties, often in the optimization literature one deals with specific classes of optimization problems, such as convex optimization [9] or linear programming [10]. In this paper, we will focus on optimization problems that satisfy (to different degrees) a Polyak-Łojasiewisc inequality (PŁI), also known as the gradient dominance condition [11], [12]. Furthermore, satisfying a PŁI globally (g PŁI) also guarantees strong robustness properties [2]. However, characterizing such a condition might not be possible for every optimization problem. In [1] the authors noticed that more general conditions than the gPŁI can be proposed by using different classes of comparison functions so that different robustness results can be guaranteed. For the discrete-time version of the problem, in [14]- [16] the authors show that it satisfies a gPŁI, guaranteeing exponential convergence to the optimal feedback law for initialization in the stabilizing set of feedback matrices.