Semi-parametric Order-based Generalized Multivariate Regression

In this paper, we consider a generalized multivariate regression problem where the responses are monotonic functions of linear transformations of predictors. We propose a semi-parametric algorithm based on the ordering of the responses which is invariant to the functional form of the transformation function. We prove that our algorithm, which maximizes the rank correlation of responses and linear transformations of predictors, is a consistent estimator of the true coefficient matrix. We also identify the rate of convergence and show that the squared estimation error decays with a rate ofo(1/ n). We then propose a greedy algorithm to maximize the highly non-smooth objective function of our model and examine its performance through extensive simulations. Finally, we compare our algorithm with traditional multivariate regression algorithms over synthetic and real data. Let us rewrite (2) in matrix form: Y n q U (X n pB p q E n q), (3) where p is the number of predictors,q is the number of responses, andn denotes the number of instances.x T i, y T i, and T i correspond, respectively, to thei -th rows ofX, Y, and E . To findB, we propose to solve: B n arg max B 1 n( q 2) n i 1 q j 1 q k 11 (y ij y ik)1 (x T i b j x T i b k) ︸ ︷︷ ︸ S n ( B), (4) whereb j denotes the j -th column ofB . The intuition behind this formulation is that sinceU is increasing and the error is i.i.d. and independent ofx, when we havex T i b j x T i b k, it is more likely to havey ij y ik than the other way around. The term in the summation is zero forj k . Maximizing S n(B) is equivalent to maximizing the average rank correlation ofy T i and x T i B since 2 S n(B) 1 corresponds to the average over then observations of the Kendall rank correlation betweeny T i and x T i B . 2. Motivating Examples and Related Work 2.1. Learning from nonlinear measurements In many practical settings, the measurements or observations are noisy nonlinear functions of a linear transformation of an underlying signal. This could be due to the uncertainties and non-linearities of the measurement device or arise from the experimental design (e.g., censoring or quantization).