Learning monotonic models with respect to a subset of the inputs is a desirable feature to effectively address the fairness, interpretability, and generalization issues in practice. Existing methods for learning monotonic neural networks either require specifically designed model structures to ensure monotonicity, which can be too restrictive/complicated, or enforce monotonicity by adjusting the learning process, which cannot provably guarantee the learned model is monotonic on selected features. In this work, we propose to certify the monotonicity of the general piece-wise linear neural networks by solving a mixed integer linear programming problem. This provides a new general approach for learning monotonic neural networks with arbitrary model structures. Our method allows us to train neural networks with heuristic monotonicity regularizations, and we can gradually increase the regularization magnitude until the learned network is certified monotonic. Compared to prior work, our method does not require human-designed constraints on the weight space and also yields more accurate approximation.

Weaknesses: I'd probably be OK with the paper being published as is, but I feel strongly that the authors should clarify one key part of their algorithm. I would be happy to increase my rating if they could make this clarification. The objective function presented in eq. As the authors themselves say, if R(f) equals 0 in the true, exact, analytic sense of the expectation w.r.t the uniform distribution over X, then the network is indeed monotonic (aside from pathological measure-0 scenarios which we will leave aside). But if it is indeed truly monotonic, then you would not need to verify it with the MILP solver.

The reviewers uniformly agreed that this is a well-written paper, on an important problem, describing a novel approach, with a good experimental evaluation. Most of their remaining concerns appear to require relatively minor changes. Of these, the most significant are (i) missing related work (R5 included some potential citations), (ii) an unclear explanation of Table 5, and (iii) that R(f) is estimated from samples, but this isn't made clear in the main text (instead of the appendix). Overall, this is a solid paper, and a few tweaks would make it even better. Please carefully read the reviews, and take their suggestions seriously when making edits.

Learning monotonic models with respect to a subset of the inputs is a desirable feature to effectively address the fairness, interpretability, and generalization issues in practice. Existing methods for learning monotonic neural networks either require specifically designed model structures to ensure monotonicity, which can be too restrictive/complicated, or enforce monotonicity by adjusting the learning process, which cannot provably guarantee the learned model is monotonic on selected features. In this work, we propose to certify the monotonicity of the general piece-wise linear neural networks by solving a mixed integer linear programming problem. This provides a new general approach for learning monotonic neural networks with arbitrary model structures. Our method allows us to train neural networks with heuristic monotonicity regularizations, and we can gradually increase the regularization magnitude until the learned network is certified monotonic.