Energy and Uncertainty: Models and Algorithms for Complex Energy Systems

I highlight several of these applications, using a simple energy storage problem as a case application. Using this setting, I describe a modeling framework that is based on five fundamental dimensions and that is more natural than the standard canonical form widely used in the reinforcement learning community. The framework focuses on finding the best policy, where I identify four fundamental classes of policies consisting of policy function approximations (PFAs), cost function approximations (CFAs), policies based on value function approximations (VFAs), and look-ahead policies. There is the familiar array of decisions: discrete actions, continuous controls, and vector-valued (and possibly integer) decisions. The tools for these problems are drawn from computer science, engineering, applied math, and operations research.