Data-Assimilation-Assisted Reinforcement Learning for Power Grid Control under Load Uncertainty

Ruoyu Hu; Feng Bao; Zezhong Zhang; Yanzhao Cao; Guannan Zhang

Home/Archive/arXiv

arXiv · 2609.27229

Data-Assimilation-Assisted Reinforcement Learning for Power Grid Control under Load Uncertainty

Ruoyu Hu·Feng Bao·Zezhong Zhang·Yanzhao Cao·Guannan Zhang

View original source ↗Read source document ↗Export metadata ↓

Abstract

Reliable power grid control requires sequential decisions under transmission constraints, time-varying demand, renewable variability, and imperfect load information. We investigate how load information quality affects control in a customized transmission network simulator inspired by Grid2Op and based on a DC power flow model. The environment includes stochastic spatial loads, temporally correlated renewable-like generation, energy storage, line protection, generator dispatch, and line reconnection. An Ensemble Score Filter (EnSF) corrects noisy load information before it is passed to the controller. A heuristic study first isolates the effect of this correction under a fixed feedback rule. Proximal policy optimization (PPO) agents are then trained for sequential generator and reconnection control, frozen, and evaluated with Forward, EnSF-corrected, and Truth load information on paired stochastic scenarios. EnSF consistently lowers load estimation error. Under heuristic control, the correction extends survival and reduces warning and overload exposure. With frozen PPO policies, it increases cumulative return and keeps performance closer to the Truth benchmark, with a larger advantage when load uncertainty is amplified beyond the nominal training distribution. In both experiments, EnSF reduces load-estimation error and improves the resulting control performance in the simplified transmission network.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Ruoyu Hu, Feng Bao, Zezhong Zhang, Yanzhao Cao, Guannan Zhang. 2026-09-23. Data-Assimilation-Assisted Reinforcement Learning for Power Grid Control under Load Uncertainty. https://arxiv.org/abs/2609.27229

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Riemannian Convex Bundle Method

We introduce the convex bundle method to solve convex, non-smooth optimization problems on Riemannian manifolds of bounded sectional curvature. Each step of our method is based on a model that involves the convex hull of previously collected subgradients, parallelly transported into the current serious iterate. This approach generalizes the dual form of classical bundle subproblems in Euclidean space. We prove that, under mild conditions, the convex bundle method converges to a minimizer. Several numerical examples implemented using Manopt$.$jl illustrate the performance of the proposed method and compare it to the subgradient method, the cyclic proximal point algorithm, as well as the proximal bundle method.

math.OC↗

arXiv2026

Omega-Limit Sets and Input-to-State Stability in Power Grids With Switching Equilibria

This paper studies a power transmission system with both conventional generators (CGs) and distributed energy assets (DEAs) providing frequency control. We consider an operating condition with demand aggregating two dynamic components: one that switches between different values on a finite set, and one that varies smoothly over time. Such dynamic operating conditions may result from protection scheme activations, external cyber-attacks, or due to the integration of dynamic loads, such as data centers. Mathematically, the dynamics of the resulting system are captured by a system that switches between a finite number of vector fields -- or modes--, with each mode having a distinct equilibrium point induced by the demand aggregation. To analyze the stability properties of the resulting switching system, we leverage tools from hybrid dynamic inclusions and the concept of $Ω$-limit sets from sets. Specifically, we characterize a compact set that is semi-globally practically asymptotically stable under the assumption that the switching frequency and load variation rate are sufficiently slow. For arbitrarily fast variations of the load, we use a level-set argument with multiple Lyapunov functions to establish input-to-state stability of a larger set and with respect to the rate of change of the loads. The theoretical results are illustrated via numerical simulations on the IEEE 39-bus test system.

math.OC↗

arXiv2026

Cellular flow control design for mixing based on the least action principle

We consider a novel approach for the enhancement of fluid mixing via pure stirring strategies building upon the Least Action Principle (LAP) for incompressible flows. The LAP is formally analogous to the Benamou--Brenier formulation of optimal transport, but imposes an incompressibility constraint. Our objective is to find a velocity field, generated by Hamiltonian flows, that minimizes the kinetic energy while ensuring that the initial scalar distribution reaches a prescribed degree of mixedness by a finite time. This formulation leads to a ``point-to-set" type of optimization problem which relaxes the requirement on controllability of the system compared to the classic LAP framework. In particular, we assume that the velocity field is induced by a finite set of cellular flows that can be controlled in time. To establish finite time feasibility, we introduce an operator-theoretic switching argument that combines the long-time cellular flow mixing result with the von Neumann alternating-projection theorem. We then leverage the direct method to establish the existence of an optimal solution. Finally, we derive the corresponding optimality conditions for the time-dependent control problem and conduct numerical experiments demonstrating the effectiveness of the proposed control design.

math.OC↗