Non-asymptotic calibration and resolution

We consider the problem of forecasting a new observation from the available data, which may include, e.g., all or some of the previous observation s and the values of some explanatory variables. To make the process of fore casting more vivid, we imagine that the data and observations are chosen by a play er called Reality and the forecasts are made by a player called Forecaster. T o establish properties of forecasting algorithms, the traditional theory of m achine learning makes some assumptions about the way Reality generates the ob servations; e.g., statistical learning theory [28] assumes that the data and obs ervations are generated independently from the same probability distribution. A m ore recent approach, prediction with expert advice (see, e.g., [5]), replaces th e assumptions about Reality by a comparison class of prediction strategies; a typical result of this theory asserts that Forecaster can perform almos t as well as the best strategies in the comparison class. This paper further explor es a third possibility, suggested in [11], which requires neither assumptions abo ut Reality nor a comparison class of Forecaster's strategies.