Reinforcement Learning for Optimal Frequency Control: A Lyapunov Approach (Papers Track) Spotlight
Wenqi Cui (University of Washington); Baosen Zhang (University of Washington)
Abstract
Renewable energy resources play a vital role in reducing carbon emissions and are becoming increasingly common in the grid. On one hand, they are challenging to integrate into a power system because the lack of rotating mechanical inertia can lead to frequency instabilities. On the other hand, these resources have power electronic interfaces that are capable of implementing almost arbitrary control laws. To design these controllers, reinforcement learning has emerged as a popular method to search for policy parameterized by neural networks. The key challenge with learning based approaches is enforcing the constraint that the learned controller need to be stabilizing. Through a Lyapunov function, we explicitly identify the structure of neural network-based controllers such that they guarantee system stability by design. A recurrent RL architecture is used to efficiently train the controllers and they outperform other approaches as demonstrated by simulations.