Shared active subspace for multivariate vector-valued functions

Many problems in machine learning, optimization, uncertainty quantification and sensitivity analysis suffer from the curse of dimensionality, where the performance and the complexity of the model worsens dramatically with the number of input variables. To alleviate this problem, one is interested in dimensionality reduction techniques. For instance, in machine learning, variable/feature selection methods Guyon and Elisseeff (2003) aim to find a subset of variables so as to improve the predictive performance of a learning algorithm, and in some algorithms, such as decision trees, the variable selection is an inherent part of the learning process. The field of sensitivity analysis mostly deals with identifying the subset of inputs parameters whose uncertainty contributes significantly to that of the model output Saltelli et al. (2008); Da Veiga et al. (2021). They are focused on the effects of the initial variables and their interactions. However, it might be the case that the model or function of interest varies the most along directions not aligned with the coordinate axes. The widely used dimensionality reduction method of principal component analysis (PCA) (also Karhunen-Loeve method) can be used to find a linear subspace of the input/output space containing the most of its variance, but, by default, it does not take into account the input-output relationship. In ecological sciences, the redundancy analysis applies PCA to the fitted values from a linear regression model to identify a subset of input parameters contributing significantly to the variation in the response matrix Legendre et al. (2011).