In Appendix A we provide more discussions on A bounds including detailed algorithm and complexity analysis comparison of different A implementations and also a small numerical

In Appendix B, we provide proofs of the theorems. In Table 6, we provide a list of oracle functions of three basic operation types, including affine transformation, unary nonlinear function, and binary nonlinear function. This lower bound can be used for training ReLU networks with loss fusion. Figure 4 compares the linear bounds in LiRP A and IBP respesctively. We refer readers to those existing works for details.