Neural Information Processing Systems http://nips.cc/

The graphical Lasso (GLASSO) is a widely used algorithm for learning high-dimensional undirected Gaussian graphical models (GGM). Given i.i.d. observations from a multivariate normal distribution, GLASSO estimates the precision matrix by maximizing the log-likelihood with an \ell_1-penalty on the off-diagonal entries. However, selecting an optimal regularization parameter λin this unsupervised setting remains a significant challenge. A well-known issue is that existing methods, such as out-of-sample likelihood maximization, select a single global λand do not account for heterogeneity in variable scaling or partial variances. Standardizing the data to unit variances, although a common workaround, has been shown to negatively affect graph recovery. Addressing the problem of nodewise adaptive tuning in graph estimation is crucial for applications like computational neuroscience, where brain networks are constructed from highly heterogeneous, region-specific fMRI data. In this work, we develop Locally Adaptive Regularization for Graph Estimation (LARGE), an approach to adaptively learn nodewise tuning parameters to improve graph estimation and selection. In each block coordinate descent step of GLASSO, we augment the nodewise Lasso regression to jointly estimate the regression coefficients and error variance, which in turn guides the adaptive learning of nodewise penalties. In simulations, LARGE consistently outperforms benchmark methods in graph recovery, demonstrates greater stability across replications, and achieves the best estimation accuracy in the most difficult simulation settings. We demonstrate the practical utility of our method by estimating brain functional connectivity from a real fMRI data set.

Estimating graphical model structure from high-dimensional and undersampled data is a fundamental problem in many scientific fields. Existing approaches, such as GLASSO, latent variable GLASSO, and latent tree models, suffer from high computational complexity and may impose unrealistic sparsity priors in some cases. We introduce a novel method that leverages a newly discovered connection between information-theoretic measures and structured latent factor models to derive an optimization objective which encourages modular structures where each observed variable has a single latent parent. The proposed method has linear stepwise computational complexity w.r.t. the number of observed variables. Our experiments on synthetic data demonstrate that our approach is the only method that recovers modular structure better as the dimensionality increases. We also use our approach for estimating covariance structure for a number of real-world datasets and show that it consistently outperforms state-of-the-art estimators at a fraction of the computational cost. Finally, we apply the proposed method to high-resolution fMRI data (with more than 10^5 voxels) and show that it is capable of extracting meaningful patterns.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. Authors propose a method of estimating a graphical model for continuous data that blends the following three, established ideas: 1) assume the data follows a multivariate Gaussian and estimate using the graphical lasso; 2) do not assume the data follows a multivariate Gaussian and instead use a Gaussian copula, the nonparanormal, to allow arbitrary single variable marginals; or 3) assume a specific tree-structured factorization and model arbitrary bivariate marginals along the tree structure. The proposed method introduces the blossom tree, which is a specific factorization of the model into a collection of densely connected blossom components that are connected by a specific set of tree edges. In particular, each blossom is connected (via a pedicel node) to at most one tree edge. The blossom components are modeled as sparse multivariate Gaussians (or using the non-paranormal copula) and the tree edges are modeled as arbitrary bivariate distributions with single variable marginals that are consistent with the marginal of any blossom pedicel to which they are attached.

Neural Information Processing Systems http://nips.cc/

We also use our approach for estimating covariance structure for a number of real-world datasets and show that it consistently outperforms state-of-the-art estimators at a fraction of the computational cost.

Identifying the Markov properties or conditional independencies of a collection of random variables is a fundamental task in statistics for modeling and inference. Existing approaches often learn the structure of a probabilistic graphical model, which encodes these dependencies, by assuming that the variables follow a distribution with a simple parametric form. Moreover, the computational cost of many algorithms scales poorly for high-dimensional distributions, as they need to estimate all the edges in the graph simultaneously. In this work, we propose a scalable algorithm to infer the conditional independence relationships of each variable by exploiting the local Markov property. The proposed method, named Localized Sparsity Identification for Non-Gaussian Distributions (L-SING), estimates the graph by using flexible classes of transport maps to represent the conditional distribution for each variable. We show that L-SING includes existing approaches, such as neighborhood selection with Lasso, as a special case. We demonstrate the effectiveness of our algorithm in both Gaussian and non-Gaussian settings by comparing it to existing methods. Lastly, we show the scalability of the proposed approach by applying it to high-dimensional non-Gaussian examples, including a biological dataset with more than 150 variables.

In this framework, the entries in Z are within [-1, 1]. In the context of stochastic block model, there is also an assignment matrix Z with entries {0, 1} (see Rohe et al 2011 Spectral clustering and the high-dimensional stochastic blockmodel). How does Z here (in the special binary case) compare with the Z matrix in Rohe et al. 2011? Why is it not allowed for all variables to be assigned to one block? It's possible that one wants to analyze the interaction network among genes from the same pathway (the same block).

Estimating graphical model structure from high-dimensional and undersampled data is a fundamental problem in many scientific fields. Existing approaches, such as GLASSO, latent variable GLASSO, and latent tree models, suffer from high computational complexity and may impose unrealistic sparsity priors in some cases. We introduce a novel method that leverages a newly discovered connection between information-theoretic measures and structured latent factor models to derive an optimization objective which encourages modular structures where each observed variable has a single latent parent. The proposed method has linear stepwise computational complexity w.r.t. the number of observed variables. Our experiments on synthetic data demonstrate that our approach is the only method that recovers modular structure better as the dimensionality increases.

Sparse estimation for Gaussian graphical models is a crucial technique for making the relationships among numerous observed variables more interpretable and quantifiable. Various methods have been proposed, including graphical lasso, which utilizes the $\ell_1$ norm as a regularization term, as well as methods employing non-convex regularization terms. However, most of these methods approximate the $\ell_0$ norm with convex functions. To estimate more accurate solutions, it is desirable to treat the $\ell_0$ norm directly as a regularization term. In this study, we formulate the sparse estimation problem for Gaussian graphical models using the $\ell_0$ norm and propose a method to solve this problem using the Difference of Convex functions Algorithm (DCA). Specifically, we convert the $\ell_0$ norm constraint into an equivalent largest-$K$ norm constraint, reformulate the constrained problem into a penalized form, and solve it using the DC algorithm (DCA). Furthermore, we designed an algorithm that efficiently computes using graphical lasso. Experimental results with synthetic data show that our method yields results that are equivalent to or better than existing methods. Comparisons of model learning through cross-validation confirm that our method is particularly advantageous in selecting true edges.