Sampling from Constrained Gibbs Measures: with Applications to High-Dimensional Bayesian Inference