Squeezing the Most Utility from Your Models

In a previous article we discussed why it's a good idea to prefer probability models to "hard" classification models, and why you should delay setting "hard" classification rules as long as possible. But decisions have to be made, and eventually you will have to set that threshold. A good threshold balances classifier precision/recall or sensitivity/specificity in a way that best meets the project or business needs. One way to quantify and think about this balance is the notion of model utility, which maps the performance of a model to some notion of the value achieved by that performance. In this article, we demonstrate the use of sigr::model_utility() to estimate model utility and pick model thresholds for classification problems.