Sparse Estimation of Inverse Covariance and Partial Correlation Matrices via Joint Partial Regression

Two important and closely related problems in statistical learning are the problems of estimating a partial correlation network and the inverse covariance matrix, also known as the precision matrix, from data. Partial correlation networks, which generalize the Gaussian graphical model, are used to model the relationships between variables while conditioning on all other variables, and are useful for inferring causal relationships between variables. Partial correlation networks are used in a plethora of applications, such as in the analysis of gene expression data, where the goal is to infer the regulatory relationships between genes (de la Fuente et al., 2004), and psychological data, where networks are used to model the relationships between psychological variables such as mood and attitude (Epskamp and Fried, 2018). The precision matrix, from which we can obtain the partial correlation network, is also of interest in its own right, as it also appears in linear discriminant analysis (Hastie et al., 2009) and in Markowitz portfolio selection (Markowitz, 1952). However, due to the high-dimensionality of the problem, estimating a precision or partial correlation matrix is often challenging as the number of parameters are on the order of the squared number of features. For this reason, classical methods such as using the inverse of the sample covariance matrix, are known to perform poorly whenever the number of observation is not extremely large. Additionally they produce estimates which are almost surely dense. This makes regularization crucial, since in many applications we typically only have a moderate number of observations, and in particular we are most often seeking a sparse estimate which gives rise to a more parsimonious and interpretable network model.