Real-time scheduling of renewable power systems through planning-based reinforcement learning

These authors contributed equally to this work. Abstract The growing renewable energy sources have posed significant challenges to traditional power scheduling. It is difficult for operators to obtain accurate day-ahead forecasts of renewable generation, thereby requiring the future scheduling system to make real-time scheduling decisions aligning with ultra-short-term forecasts. Restricted by the computation speed, traditional optimization-based methods can not solve this problem. Recent developments in reinforcement learning (RL) have demonstrated the potential to solve this challenge. However, the existing RL methods are inadequate in terms of constraint complexity, algorithm performance, and environment fidelity. The proposed approach enables planning and finer time resolution adjustments of power generators, including unit commitment and economic dispatch, thus increasing the grid's ability to admit more renewable energy. The well-trained scheduling agent significantly reduces renewable curtailment and load shedding, which are issues arising from traditional scheduling's reliance on inaccurate day-ahead forecasts. High-frequency control decisions exploit the existing units' flexibility, reducing the power grid's dependence on hardware transformations and saving investment and operating costs, as demonstrated in experimental results. This research exhibits the potential of reinforcement learning in promoting low-carbon and intelligent power systems and represents a solid step toward sustainable electricity generation. Climate change and carbon neutrality have garnered widespread global attention. The significant amount of carbon emissions during electricity production underscores the importance of achieving low-carbon electricity production as a solution to these pressing challenges. In recent years, wind and solar energy have emerged as promising sources of sustainable electricity. However, the fluctuation patterns of these sources are highly variable, making it challenging to accurately predict their power generation capacity over the long term. This presents a major challenge for existing power scheduling systems that rely on reliable long-term forecasts and day-ahead calculation, potentially leading to suboptimal or infeasible solutions, including renewable curtailments and blackouts [1, 2]. Traditionally, power system operators perform the day-ahead scheduling (DAS) program to calculate power generation schedules [3].