Welcome to Pushing Buttons, the Guardian's gaming newsletter. If you'd like to receive it in your inbox every week, just pop your email in below – and check your inbox (and spam) for the confirmation email. There's been an interesting development in the games business this week: Square Enix, the Japanese company behind Final Fantasy, has sold off basically its entire North American business for $300m. Swedish entrepreneur collective Embracer Group, a relative newcomer in gaming, is now the proud owner of studios in Montreal the US, and properties like Deus Ex, Thief and, of course, Tomb Raider. Not too long ago, this would have felt like big news purely because of the money involved.

Reinforcement learning (RL) is empirically successful in complex nonlinear Markov decision processes (MDPs) with continuous state spaces. By contrast, the majority of theoretical RL literature requires the MDP to satisfy some form of linear structure, in order to guarantee sample efficient RL. Such efforts typically assume the transition dynamics or value function of the MDP are described by linear functions of the state features. To resolve this discrepancy between theory and practice, we introduce the Effective Planning Window (EPW) condition, a structural condition on MDPs that makes no linearity assumptions. We demonstrate that the EPW condition permits sample efficient RL, by providing an algorithm which provably solves MDPs satisfying this condition. Our algorithm requires minimal assumptions on the policy class, which can include multi-layer neural networks with nonlinear activation functions. Notably, the EPW condition is directly motivated by popular gaming benchmarks, and we show that many classic Atari games satisfy this condition. We additionally show the necessity of conditions like EPW, by demonstrating that simple MDPs with slight nonlinearities cannot be solved sample efficiently.