Gaussian graphical models (GGMs), which represent the dependence structure among a set of random variables, have been widely used to model the conditional dependence relationships in many applications, including gene regulatory networks and brain connectivity maps (Drton and Maathuis, 2017; Varoquaux et al., 2010; Zhao et al., 2014; Glymour et al., 2019). In the classical setting with data from a single study, the estimation of high-dimensional GGMs has been well studied in a series of papers, including penalized likelihood methods (Yuan and Lin, 2007; Lam and Fan, 2009; Friedman et al., 2008; Rothman et al., 2008) and convex optimization based methods (Cai et al., 2011, 2016; Liu and Wang, 2017). The minimax optimal rates are studied in Cai et al. (2016) and a review can be found in Cai (2017). Liu (2013) considers the inference in GGMs based on a node-wise regression approach and Ren et al. (2015) studies the estimation optimality and inference for individual entries. Methods for estimating a single GGM have also been extended to simultaneously estimating multiple graphs when data from multiple studies are available. For example, Guo et al. (2011); Danaher et al. (2014); Cai et al. (2016) consider jointly estimating multiple GGMs with some penalties for inducing common structures among different graphs.