Process Knowledge Driven Change Point Detection for Automated Calibration of Discrete Event Simulation Models Using Machine Learning

Initial development and subsequent calibration of discrete event simulation models for complex systems require accurate identification of dynamically changing process characteristics. Existing data driven change point methods (DD-CPD) assume changes are extraneous to the system, thus cannot utilize available process knowledge. This work proposes a unified framework for process-driven multivariate change point detection (PD-CPD) by combining change point detection models with machine learning and process-driven simulation modeling. The PD-CPD, after initializing with DD-CPD's change point(s), uses simulation models to generate system level outputs as time-series data streams which are then used to train neural network models to predict system characteristics and change points. The accuracy of the predictive models measures the likelihood that the actual process data conforms to the simulated change points in system characteristics. PD-CPD iteratively optimizes change points by repeating simulation and predictive model building steps until the set of change point(s) with the maximum likelihood is identified. Using an emergency department case study, we show that PD-CPD significantly improves change point detection accuracy over DD-CPD estimates and is able to detect actual change points. This work has been submitted to the IEEE for possible publication. Increasing complexity of modern systems require a new generation of simulation models that can accurately represent the time-varying system dynamics and provide decision support. In manufacturing and service systems, discrete event simulation (DES) models are extensively used to represent the discrete flow of materials, requests and customers in dynamic environments. Challenges associated with the complex nature of modern systems are being increasingly addressed by digital twin technologies [1], [2] that aim to create a one-to-one replica of the physical world using highly detailed simulation models.