A Deep Q-Learning based Smart Scheduling of EVs for Demand Response in Smart Grids

However, the rapid adoption of small-scale renewable at the edge of the grid makes the smart grid management process more complex and exposed to uncertainty related to renewable production [1, 2]. The digitization and decentralization principles bring to the forefront the energy demand flexibility as a key support to accommodate high shares of variable renewable energy [3, 4]. Leveraging local flexibility is possible to maintain a balance between supply and demand at lower costs using the energy assets of the citizens rather than the ones owned by the grid operator that are more expensive to operate [5-7]. The challenges are even more evident and difficult to tackle in the context of the increased adoption of electrical vehicles (EVs) [8]. In that respect grid management should closely cooperate and interact within a low latency context with EVs coordination and aggregation services to procure their energy scheduling flexibility to support local network balancing or to achieve self-sufficiency [9, 10]. However, EVs usage has several shortcomings such as the limited battery range, relatively short battery lifespan, averaging 10-20 years or up to 150,000 miles, and the lack of existing infrastructure for charging electric vehicles. Other major issues refer to the impact of EVs on the power grid, encompassing factors such as the rise in short-circuit currents, deviations in voltage levels beyond standard limits, and the potential impact on the lifespan of equipment due to increased power demand [11].