Thisresult shows that visual grounding/feature attribution can be ineffective at resolving shortcuts.

As currently described, we can only apply our method to block design, fully observed settings where alldatapoints andtasks areobservedatthesame time.

A.1 ProofsofSection3 In the following, the symbolh, i will be used to represent both Frobenius norm of matrices and standard Euclidean norm of vectors. For the second part, letv be an eigenvector ofA corresponding to eigenvalueλmax(A). Incase the maximum eigenvalue ofT isnonpositive, then from Lemma A.1 we see that the objectivevalue of problem(A.2)evaluated For anp preal matrixA, its spectral radiusR(A)is defined as the largest absolute value of its eigenvalues. Then the matrixI A is invertible and all entries of(I A) 1 are nonnegative. Also the spectral radius of(γ?) 1bΩ12V(β)bΩ12 is smaller than1 by the feasibility ofγ? in problem (A.5c).

Ontheother hand, the reference set could also be a (good) approximated Pareto front for assessment. In this paper,we use the exact Pareto front for MOKP(3-100) tocompute IGD since theyare easy tobe solvedoptimally.