Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"

A recent paper [1] by Liu et al. combines the topics of adversarial training and Bayesian Neural Networks (BNN) and suggests that adversarially trained BNNs are more robust against adversarial attacks than their non-Bayesian counterparts. Here, I analyze the proposed defense and suggest that one needs to adjust the adversarial attack to incorporate the stochastic nature of a Bayesian network to perform an accurate evaluation of its robustness. Using this new type of attack I show that there appears to be no strong evidence for higher robustness of the adversarially trained BNNs. The evaluation of a neural network has proven to be a complex and difficult task, as one needs to separate two causes for the same observation - the robustness of the defended network and the shortcomings of the attack. If a network appears to be robust, this can either mean that it is in fact robust against adversarial attacks or that the attack is incomplete or relies on inapplicable assumptions on the attacked network. Recently, a new paper [1] applied adversarial training [2] on Bayesian Neural Networks(BNNs).