Tutorial on Implied Posterior Probability for SVMs

Department of Data Science, Medical Data Science Ltd., Bulgaria Editor: Abstract Implied posterior probability of a given model (say, Support Vector Machines (SVM)) at a point x is an estimate of the class posterior probability pertaining to the class of functions of the model applied to a given dataset. It can be regarded as a score (or estimate) for the true posterior probability, which can then be calibrated/mapped onto expected (non-implied by the model) posterior probability implied by the underlying functions, which have generated the data. In this tutorial we discuss how to compute implied posterior probabilities of SVMs for the binary classification case as well as how to calibrate them via a standard method of isotonic regression. Keywords: Posterior probability, Bayes rule, Classification, SVMs 1. Introduction The implied posterior probability method for estimating class posterior probability has recently been proposed (Nalbantov and Ivanov, 2019). The method provides a score (or estimate) for the true posterior probability, which can then be calibrated/mapped onto expected (non-implied by the model) posterior probability implied by the underlying functions, which have generated the data. The main difference with other methods for solving this problem is the non-reliance on the original model built on the data to estimate posterior probabilities for points which do not belong to the separation surface of the model. Rather, the estimates are based on the class of functions used to build the (original) model, as applied to different versions of the dataset, where the relative weight of the instances varies between the classes. For each such relative weight a different model is built, which is relevant for the estimation of a particular value of the posterior probability.