On parameters transformations for emulating sparse priors using variational-Laplace inference

So-called sparse estimators arise in the context of model fitting, when one a priori assumes that only a few (unknown) model parameters deviate from zero (Li, 2007). Typically, sparsity constraints can be useful when the estimation problem is under-determined, i.e. when number of parameters to estimate ( This is why alternative approaches have been proposed, such as the so-called LASSO estimator (Tibshirani, 1996), which stands for Least Absolute Shrinkage and Selection Operator. Other alternative methods include, e.g., so-called "elastic nets", which use a mixture of l 1 and l Zou and Hastie, 2005), and "Horseshoe estimators", which are Bayesian estimators relying on mixture of normal priors (Carvalho et al., 2010). Note that, from a Bayesian perspective, sparsity always derives from the "fat tails" of effective priors that eventually yield the regularized estimate (Griffin and Brown, 2013). We then demonstrate the approach using Monte-Carlo simulations.