Sampling with Trustworthy Constraints: A Variational Gradient Framework