Summary and Contributions: Update: I've read this paper many times, and I have always had a lot of trouble understanding the mathematical development leading to the objective function. I now understand it better, so I'd like to suggest how I would present it, in case it gives you some ideas for your own presentation: eps(x,y) is the error between y and the "closest mode". Let's define m(x,y) to be a deterministic "mode function" that returns the mode of p(y x) that is closest to y. By modeling assumption, we assert that for fixed x and Y p(y x), we have eps(x,Y) N(0,sig 2). We want to approximate the function eps(x,y) with a function from the class f_theta(x,y).

Thank you for your submission to NeurIPS. This was a borderline paper, with three of the reviewers being position (including one that went from negative to positive after the discussion period), and one remaining quite unconvinced about the paper. Having read through the paper myself, I agree with the overall assessment that there are both some strong and weak aspects to the paper. On the positive side, I found the proposed modeling approach here genuinely interesting. The idea of parameterizing a kind of energy function (I know the authors don't use this notion, but I do feel it is warranted ... this refers to the loss function that is ultimately minimized to find the modes) as the combination of the squared value of a the function f plus the penalty of the function derivative away from zero, this was rather interesting, and I believe likely to have applications well beyond just modal regression.

For multi-valued functions---such as when the conditional distribution on targets given the inputs is multi-modal---standard regression approaches are not always desirable because they provide the conditional mean. Modal regression algorithms address this issue by instead finding the conditional mode(s). Most, however, are nonparametric approaches and so can be difficult to scale. Further, parametric approximators, like neural networks, facilitate learning complex relationships between inputs and targets. In this work, we propose a parametric modal regression algorithm. We use the implicit function theorem to develop an objective, for learning a joint function over inputs and targets.

For multi-valued functions---such as when the conditional distribution on targets given the inputs is multi-modal---standard regression approaches are not always desirable because they provide the conditional mean. Modal regression aims to instead find the conditional mode, but is restricted to nonparametric approaches. Such methods can be difficult to scale, and cannot benefit from parametric function approximation, like neural networks, which can learn complex relationships between inputs and targets. In this work, we propose a parametric modal regression algorithm, by using the implicit function theorem to develop an objective for learning a joint parameterized function over inputs and targets. We empirically demonstrate on several synthetic problems that our method (i) can learn multi-valued functions and produce the conditional modes, (ii) scales well to high-dimensional inputs and (iii) is even more effective for certain uni-modal problems, particularly for high frequency data where the joint function over inputs and targets can better capture the complex relationship between them. We then demonstrate that our method is practically useful in a real-world modal regression problem. We conclude by showing that our method provides small improvements on two regression datasets that have asymmetric distributions over the targets.