Variable projection without smoothness

R is smooth, and h and r are convex (but possibly non-smooth). In particular, we target applications in signal-processing, high-dimensional statistics and machine learning. Here, x is viewed as a variable of primary interest, while θ represents a set of auxiliary (nuisance) parameters. In many applications, efficient algorithms have been developed to globally minimize the objective function in x for fixed θ. We provide a provably convergent algorithmic recipe to extend these algorithms to (1). We begin by reviewing the classic Variable Projection (VP) technique for nonlinear least squares problems. Early work on the topic [10] has found numerous applications in chemistry, mechanical systems, neural networks, and telecommunications (see the surveys of [11] and [15], and references therein.)