SMML estimators for exponential families with continuous sufficient statistics

The minimum message length(MML) principle[7] is an information theoretic criterion that links data compression with statistical inference [6]. It has a number of useful properties and it has close connections with Kolmogorov complexity [8]. Using the MML principle to construct estimators is known to be NPhard in general [4] so it is common to use approximations in practice [6]. The term'strict minimum message length' (SMML) is used for the exact MML criterion, to distinguish it from the various approximations. The only known algorithm for calculating an SMML estimator is Farr's algorithm [4] which applies to data taking values in a finite set which is (in some sense) 1-dimensional. A method for calculating SMML estimators for 1-dimensional exponential families with continuous sufficient statistics was also recently given in [3]. However, calculating SMML estimators for higher-dimensional data is still a difficult problem.