Reviews: Unconstrained Monotonic Neural Networks

However, even after reading the rebuttal, I feel that it is a bit premature to publish the research at this point in time. In the rebuttal, the authors acknowledge that their method is not the first universal monotonic approximator and clarify that their language regarding the "cap on expressiveness" of alternative monotonic approximators refers to the non-asymptotic case, i.e., a finite number of neurons/hidden units. They write "we believe that the constraints on the positiveness of the weights and on the class of possible activation functions are unnecessarily restraining the hypothesis space in the non-asymptotic case". However, this is an assertion for which they have not supplied any kind of proof, and I find it highly debatable. Any method, whether it is their UMNN or the Huang approach or lattices or max/min networks, has some cap on expressiveness in the non-asymptotic case.