The Effect of Singularities in a Learning Machine when the True Parameters Do Not Lie on such Singularities

A lot of learning machines with hidden variables used in infor- mation science have singularities in their parameter spaces. At singularities, the Fisher information matrix becomes degenerate, resulting that the learning theory of regular statistical models does not hold. Recently, it was proven that, if the true parameter is contained in singularities, then the coe(cid:14)cient of the Bayes gen- eralization error is equal to the pole of the zeta function of the Kullback information. In this paper, under the condition that the true parameter is almost but not contained in singularities, we show two results.