How consistent is my model with the data? Information-Theoretic Model Check

Parametric statistical inference often begins with the choice of a model class which is used to describe an unknown datagenerating process. In system identification and sequential data analysis, we obtain a sequence of dependent samples from this process. A classical problem has been to assess whether the unknown process is contained in the proposed model class, usually relying on large-sample results (White, 1982). In many real-world applications, however, we only have a limited data record and we expect the model class to be misspecified in some respect. A more relevant question would then be: how consistent is the model class with the observed data? A classical means of assessing a model is through its residuals or prediction errors. E.g. for linear dynamic models, one can check whether their prediction errors constitute a white noise process, cf.