UNIFY: a Unified Policy Designing Framework for Solving Constrained Optimization Problems with Machine Learning

Methods for combining Machine Learning (ML) and Constrained Optimization (CO) for decision support have attracted considerable interest in recent years. This is motivated by the possibility to tackle complex decision making problems subject to uncertainty (sometimes over multiple stages), and having a partially specified structure where knowledge is available both in explicit form (cost function, constraints) and implicit form (historical data or simulators). As a practical example, an Energy Management Systems (EMS) needs to allocate minimum-cost power flows from different Distributed Energy Resources (DERs) [1]. Based on actual energy prices, and forecasts on the availability of DERs and on consumption, the EMS decides which power generators should be used and whether the surplus should be stored or sold to the market. Such a problem involves hard constraints (maintaining power balance, power flow limits), a clear cost structure, elements of uncertainty that are partially known via historical data, and multiple decision stages likely subject to execution time restrictions. In this type of use case, pure CO methods struggle with robustness and scalability, while pure ML methods such as Reinforcement Learning (RL) have trouble dealing with hard constraints and combinatorial decision spaces. Motivated by the opportunity to obtain improvements via a combination of ML and CO, multiple lines of research have emerged, such as Decision Focused Learning, Constrained Reinforcement Learning, or Algorithm Configuration. While existing methods have obtained a good measure of success, to the best of the authors knowledge no existing method can deal with all the challenges we have identified. Ideally, one wishes to obtain a solution policy capable of providing feasible (and high-quality) solutions, handling robustness, taking advantage of existing data, and with a reasonable computational load.