Tradeoffs in Streaming Binary Classification under Limited Inspection Resources

Institutions are increasingly relying on machine learning models Given the imbalanced nature of data in this domain, which makes to identify and alert on abnormal events, such as fraud, cyber attacks learning classifiers that efficiently discriminate among the minority and system failures. These alerts often need to be manually and majority class difficult, and the limited resources available investigated by specialists. Given the operational cost of manual inspections, for inspecting time-sensitive risky events, we are interested in understanding the suspicious events are selected by alerting systems with the relationship between the rate of detection from the carefully designed thresholds. In this paper, we consider an imbalanced minority class (i.e., the fraction of samples from the minority class binary classification problem, where events arrive sequentially selected for inspection) and the inspection budget. Specifically, we and only a limited number of suspicious events can be inspected. We focus on applications that involve real-time processing and decisionmaking model the event arrivals as a non-homogeneous Poisson process, and where an abnormal event can only be inspected at the time compare various suspicious event selection methods including those of arrival, and we investigate how different selection policies based based on static and adaptive thresholds. For each method, we analytically on classifier predictions operate in terms of the limited inspection characterize the tradeoff between the minority-class detection budget rather than the decision threshold.