Mean-Field Generalisation Bounds for Learning Controls in Stochastic Environments

When solving stochastic control problems, one is often limited by the challenge of specifying realistic model dynamics of the involved processes. Parametric approaches to estimating dynamics introduce model error, while'model-free' approaches typically suffer from extreme curse of dimensionality constraints. The development of reliable machine-learning based methods for stochastic control is therefore of significant practical interest. In this paper, we focus on problems where a decision maker faces a stochastic environment, that is, where they interact with a system with unknown and uncontrolled stochastic dynamics, which, together with their control, induce a controlled state process and costs. Examples of this include optimal investment for a small investor - here the stochastic dynamics of assets are uncontrolled and unknown, the investor chooses a strategy based on past observations, and together these generate a wealth process which must be optimised. A second example is aerial navigation in the presence of uncertain weather - the weather is unaffected by the navigation policy chosen, while the navigator must account for uncertainties in their planning, and the resulting flight-plan needs to be optimised. In both these cases, the stochastic environment is naturally high-dimensional and may not be Markovian, and so is challenging to model statistically using finitely many observations. We consider the setting where we have access to a finite number of i.i.d.