Using Eigencentrality to Estimate Joint, Conditional and Marginal Probabilities from Mixed-Variable Data: Method and Applications

Abstract--The ability to estimate joint, conditional and marginal probability distributions over some set of variables is of great utility for many common machine learning tasks. However, estimating these distributions can be challenging, particularly in the case of data containing a mix of discrete and continuous variables. This paper presents a nonparametric method for estimating these distributions directly from a dataset. The data are first represented as a graph consisting of object nodes and attribute value nodes. Depending on the distribution to be estimated, an appropriate eigenvector equation is then constructed. This equation is then solved to find the corresponding stationary distribution of the graph, from which the required distributions can then be estimated and sampled from. The paper demonstrates how the method can be applied to many common machine learning tasks including classification, regression, missing value imputation, outlier detection, random vector generation, and clustering. Being able to estimate joint, conditional and marginal probabilities from some dataset allows a broad range of useful tasks to be performed. For example, classification and regression involve predicting the value of some target variable conditional on the values of the other variables. If we can sample values from the estimated distributions, we could perform random vector generation by generating full random vectors that display the same correlations as the vectors (i.e., data points) in the original data [4], [5]. If we can estimate the joint distribution for the full dataset, then we should also be able to do this for subsets of data, leading to the use of Expectation-Maximization [6] to cluster the data [7]. Taken together, these activities form a large chunk of the tasks commonly used in machine learning. All of this depends, of course, on being able to estimate the various probabilities, and this is particularly challenging on datasets containing a complex mix of continuous and discrete variables.