The "black-box" nature of representations, combined with the known susceptibility of networks to learn spurious correlations [

We argue that this is due, in part, to a common feature of many interpretability methods: they analyze model behavior by using a particular dataset.

However, many use cases of neural network verification require solving the inverse problem, or over-approximating the set of inputs that lead to certain outputs.

Bayesian brain theory suggests that the brain employs generative models to understand the external world.

The following proof is from Daniely and V ardi [15], and we give it here for completeness. By Lemma A.1, there exists a DNF formula We construct such an affine layer in Lemma A.2. At least one of the k size-n slices in z contains 0 more than once. We define the outputs of our affine layer as follows. Pr [z represents a hyperedge ] = n (n 1) ... (n k + 1) null 1 n null Pr null z Z null 1 2 log(n) .

Gaussian, and the weight matrix is non-degenerate.