Supplement to The Hessian Screening Rule Jonas Wallin Department of Statistics Department of Statistics Lund University A Proofs

A.1 Proof of Theorem 1 It suffices to verify that the KKT conditions hold for ˆβ In this section we present the algorithms for efficiently updating the Hessian and its inverse (Algorithm 1) and the full algorithm for the Hessian screening method (Algorithm 2). In this section, we discuss situations in which the Hessian is singular or ill-conditioned and propose remedies for these situations. It is not, however, generally the case with discrete-valued data, particularly not in when p n. Algorithm 1 This algorithm provides computationally efficient updates for the inverse of the Hessian. Note the slight abuse of notation here in that E is used both for X and Q. A simple instance of this occurs when the columns of X are duplicates, in which case |e| = 2. Duplicated predictors are fortunately easy to handle since they enter the model simultaneously.