Some aspects of robustness in modern Markov Chain Monte Carlo