Using Knowledge Distillation to improve interpretable models in a retail banking context

Although the banking sector holds massive troves of data regarding its customers, products and transactions, and is no stranger to using quantitative tools to inform its decisions, two constraints usually weigh on the development of predictive models. The first one lies in the regulatory obligation to use interpretable models for a wide range of issues, with the management function being able to explain both the way a model was trained and why specific decisions have been made. Indeed, the European Banking Authority (2020) urges banking institutions to "understand the models used, and their methodology, input data, assumptions, limitations and outputs". The second has to do with the production environments available to deploy the models on. Due to the persistence of legacy systems, cost constraints or execution time limits -- think real time e-commerce fraud detection -- models may be limited to simple operations and conditions, i.e. a set of rules rather than a random forest, light computations in place of a fully fledged neural network. Modeling for retail banking use cases means dealing with both these strong customers protections -- enforced through regular audits -- and the high data volume which at times shortens the time allocated to each sample. These shackles help explain why modeling practices in retail banking departments are centered around simple and interpretable models such as the logistic regression or the (shallow) decision tree.