arXiv subjects

Florian Dörfler

Publications and source records attributed to Florian Dörfler.

At least 163 records · Page 9Linked to original sources

arXiv2023

Providing Curative Distribution Grid Flexibility Using Online Feedback Optimization

Distribution grid flexibility is discussed as a possible measure in curative system operation, yielding a need for an efficient and robust coordination mechanism for the joint flexibility provision by individual units to the transmission grid. This paper introduces a method to coordinate distribution grid level flexibility as a fast-responding curative measure based on Online Feedback Optimization. We utilize an optimization algorithm in a closed loop with the distribution grid to dispatch set points for active and reactive power to flexibility providing units. The approach is evaluated in an experimental setup, utilizing assets connected to an exemplary low voltage grid. Online Feedback Optimization is found to be both a viable as well as a highly effective approach to coordinate distributed energy resources in real-time curative system operation.

eess.SY↗

arXiv2023

Efficient Exploration in Continuous-time Model-based Reinforcement Learning

Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use the optimistic principle for exploration. Our regret bounds surface the importance of the measurement selection strategy(MSS), since in continuous time we not only must decide how to explore, but also when to observe the underlying system. Our analysis demonstrates that the regret is sublinear when modeling ODEs with Gaussian Processes (GP) for common choices of MSS, such as equidistant sampling. Additionally, we propose an adaptive, data-dependent, practical MSS that, when combined with GP dynamics, also achieves sublinear regret with significantly fewer samples. We showcase the benefits of continuous-time modeling over its discrete-time counterpart, as well as our proposed adaptive MSS over standard baselines, on several applications.

cs.LG↗

arXiv2023

Tutorial on Congestion Control in Multi-Area Transmission Grids via Online Feedback Equilibrium Seeking

Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control methodology is applied to multi-area transmission grids, where each area independently manages its congestion while being dynamically interconnected with the rest of the grid, a non-cooperative game arises. In this context, OFO must be interpreted as an online feedback equilibrium seeking (FES) scheme. Our analysis incorporates technical tools from game theory and monotone operator theory to evaluate the stability and performance of multi-area grid operation. Through numerical simulations, we illustrate the key challenge of this non-cooperative setting: on the one hand, independent multi-area decisions are suboptimal compared to a centralized control scheme; on the other hand, some areas are heavily penalized by the centralized decision, which may discourage participation in the coordination mechanism.

math.OC↗

arXiv2023

Controller implementability: a data-driven approach

We study the controller implementability problem, which seeks to determine if a controller can make the closed-loop behavior of a given plant match that of a desired reference behavior. We establish necessary and sufficient conditions for controller implementability which only rely on raw data. Subsequently, we consider the problem of constructing controllers directly from data. By leveraging the concept of canonical controller, we provide a formula to directly construct controllers that implement plant-compatible reference behaviors using measurements of both reference and plant behaviors.

math.OC↗

arXiv2023

Data-driven representations of conical, convex, and affine behaviors

The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on infinite-horizon measurements. We then provide necessary and sufficient conditions for representing conical, convex, and affine finite-horizon behaviors using raw data matrices, expressing persistence of excitation requirements in terms of non-negative rank conditions. The applicability of our results is demonstrated by a numerical example arising in population ecology.

math.OC↗

arXiv2023

CARMA: Fair and efficient bottleneck congestion management via non-tradable karma credits

This paper proposes a non-monetary traffic demand management scheme, named CARMA, as a fair solution to the morning commute congestion. We consider heterogeneous commuters traveling through a single bottleneck that differ in both the desired arrival time and Value of Time (VOT). We consider a generalized notion of VOT by allowing it to vary dynamically on each day (e.g., according to trip purpose and urgency), rather than being a static characteristic of each individual. In our CARMA scheme, the bottleneck is divided into a fast lane that is kept in free flow and a slow lane that is subject to congestion. We introduce a non-tradable mobility credit, named karma, that is used by commuters to bid for access to the fast lane. Commuters who get outbid or do not participate in the CARMA scheme instead use the slow lane. At the end of each day, karma collected from the bidders is redistributed, and the process repeats day by day. We model the collective commuter behaviors under CARMA as a Dynamic Population Game (DPG), in which a Stationary Nash Equilibrium (SNE) is guaranteed to exist. Unlike existing monetary schemes, CARMA is demonstrated, both analytically and numerically, to achieve a) an equitable traffic assignment with respect to heterogeneous income classes and b) a strong Pareto improvement in the long-term average travel disutility with respect to no policy intervention. With extensive numerical analysis, we show that CARMA is able to retain the same congestion reduction as an optimal monetary tolling scheme under uniform karma redistribution and even outperform tolling under a well-designed redistribution scheme. We also highlight the privacy-preserving feature of CARMA, i.e., its ability to tailor to the private preferences of commuters without centrally collecting the information.

eess.SY↗

arXiv2023

Data-enabled Policy Optimization for the Linear Quadratic Regulator

Policy optimization (PO), an essential approach of reinforcement learning for a broad range of system classes, requires significantly more system data than indirect (identification-followed-by-control) methods or behavioral-based direct methods even in the simplest linear quadratic regulator (LQR) problem. In this paper, we take an initial step towards bridging this gap by proposing the data-enabled policy optimization (DeePO) method, which requires only a finite number of sufficiently exciting data to iteratively solve the LQR problem via PO. Based on a data-driven closed-loop parameterization, we are able to directly compute the policy gradient from a batch of persistently exciting data. Next, we show that the nonconvex PO problem satisfies a projected gradient dominance property by relating it to an equivalent convex program, leading to the global convergence of DeePO. Moreover, we apply regularization methods to enhance certainty-equivalence and robustness of the resulting controller and show an implicit regularization property. Finally, we perform simulations to validate our results.

math.OC↗

arXiv2023

Nash equilibrium seeking over digraphs with row-stochastic matrices and network-independent step-sizes

In this paper, we address the challenge of Nash equilibrium (NE) seeking in non-cooperative convex games with partial-decision information. We propose a distributed algorithm, where each agent refines its strategy through projected-gradient steps and an averaging procedure. Each agent uses estimates of competitors' actions obtained solely from local neighbor interactions, in a directed communication network. Unlike previous approaches that rely on (strong) monotonicity assumptions, this work establishes the convergence towards a NE under a diagonal dominance property of the pseudo-gradient mapping, that can be checked locally by the agents. Further, this condition is physically interpretable and of relevance for many applications, as it suggests that an agent's objective function is primarily influenced by its individual strategic decisions, rather than by the actions of its competitors. In virtue of a novel block-infinity norm convergence argument, we provide explicit bounds for constant step-size that are independent of the communication structure, and can be computed in a totally decentralized way. Numerical simulations on an optical network's power control problem validate the algorithm's effectiveness.

cs.GT↗

arXiv2023

Control Design of Dynamic Virtual Power Plants: An Adaptive Divide-and-Conquer Approach

In this paper, we present a novel control approach for dynamic virtual power plants (DVPPs). In particular, we consider a group of heterogeneous distributed energy resources (DERs) which collectively provide desired dynamic ancillary services such as fast frequency and voltage control. Our control approach relies on an adaptive divide-and-conquer strategy: first, we disaggregate the desired frequency and voltage control specifications of the aggregate DVPP via adaptive dynamic participation matrices (ADPMs) to obtain the desired local behavior for each device. Second, we design local linear parameter-varying (LPV) $\mathcal{H}_\infty$ controllers to optimally match this local behaviors. In the process, the control design also incorporates the physical and engineered limits of each DVPP device. Furthermore, our adaptive control design can properly respond to fluctuating device capacities, and thus include weather-driven DERs into the DVPP setup. Finally, we demonstrate the effectiveness of our control strategy in a case study based on the IEEE nine-bus system.

eess.SY↗

arXiv2023

Distributional Uncertainty Propagation via Optimal Transport

This paper addresses the limitations of standard uncertainty models, e.g., robust (norm-bounded) and stochastic (one fixed distribution, e.g., Gaussian), and proposes to model uncertainty via Optimal Transport (OT) ambiguity sets. These constitute a very rich uncertainty model, which enjoys many desirable geometrical, statistical, and computational properties, and which: (1) naturally generalizes both robust and stochastic models, and (2) captures many additional real-world uncertainty phenomena (e.g., black swan events). Our contributions show that OT ambiguity sets are also analytically tractable: they propagate easily and intuitively through linear and nonlinear (possibly corrupted by noise) transformations, and the result of the propagation is again an OT ambiguity set or can be tightly upper bounded by an OT ambiguity set. In the context of dynamical systems, our results allow us to consider multiple sources of uncertainty (e.g., initial condition, additive noise, multiplicative noise) and to capture in closed-form, via an OT ambiguity set, the resulting uncertainty in the state at any future time. Our results are actionable, interpretable, and readily employable in a great variety of computationally tractable control and estimation formulations. To highlight this, we study three applications in trajectory planning, consensus algorithms, and least squares estimation. We conclude the paper with a list of exciting open problems enabled by our results.

math.OC↗

arXiv2023

Experimental Validation of a Dynamic Virtual Power Plant Concept Based on Multiple-Converter Power Hardware-In-the-Loop Test Bench

Recently, the concept of dynamic virtual power plants (DVPP) has been proposed to collectively provide desired dynamic ancillary services such as fast frequency and voltage control by a heterogeneous ensemble of distributed energy resources (DER). This paper presents an experimental validation of a recent DVPP control design approach on a multi-converter power hardware-in-the-loop (PHIL) test bed system. More specifically, we consider a DVPP composed of a wind generation system, a photovoltaic (PV) system, and a STATCOM with small storage capacity to collectively provide grid-following fast frequency regulation in the presence of grid-frequency and load variations. The performance of the aggregated DVPP response is evaluated with respect to its ability to match a desired dynamic behavior while taking practical limitations of the individual DVPP units into account.

eess.SY↗

arXiv2023

Deployment of an Online Feedback Optimization Controller for Reactive Power Flow Optimization in a Distribution Grid

Optimization is an essential part of power grid operation and lately, Online Optimization methods have gained traction. One such method is Online Feedback Optimization (OFO) which uses measurements from the grid as feedback to iteratively change the control inputs until they converge to the solution of the optimization problem. Such algorithms have been applied to many power system problems and experimentally validated in lab setups. This paper implements an OFO controller in a real distribution grid for 24/7 operation using off-the-shelf hardware and software. The proposed control strategy optimizes the reactive power flow at the substation while satisfying voltage constraints. As part of an existing coordination scheme between (sub)transmission grid operator (TSO) and distribution grid operator (DSO), this comes with a financial reward and simultaneously it virtually reinforces the grid by regulating the voltage on the feeder and therefore allowing higher levels of distributed generation/consumption. We present how a distribution grid is retrofitted such that we can use existing inverters, we analyze the controller's interaction with legacy infrastructure, and investigate its overall control behavior. Finally, we demonstrate the successful deployment of an OFO controller in an operational environment which corresponds to Technology Readiness Level (TRL) 7.

eess.SY↗

arXiv2023

Strategic Interactions in Multi-modal Mobility Systems: A Game-Theoretic Perspective

The evolution of existing transportation systems,mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the heterogeneous stakeholders involved in the mobility ecosystem and analyze how they influence the system. In this paper, we focus on the interactions between citizens who compete for the limited resources of a mobility system to complete their desired trip. Specifically, we present a game-theoretic framework for multi-modal mobility systems, where citizens, characterized by heterogeneous preferences, have access to various mobility options and seek individually-optimal decisions. We study the arising game and prove the existence of an equilibrium, which can be efficiently computed via a convex optimization problem. Through both an analytical and a numerical case study for the classic scenario of Sioux Falls, USA, we illustrate the capabilities of our model and perform sensitivity analyses. Importantly, we show how to embed our framework into a "larger" game among stakeholders of the mobility ecosystem (e.g., municipality, Mobility Service Providers, and citizens), effectively giving rise to tools to inform strategic interventions and policy-making in the mobility ecosystem.

cs.MA↗

arXiv2023

Grid-Forming Hybrid Angle Control: Behavior, Stability, Variants and Verification

This work explores the stability, behavior, variants, and a controller-hardware-in-the-loop (C-HiL) verification of the recently proposed grid-forming (GFM) hybrid angle control (HAC). We revisit the foundation of GFM HAC, and highlight its behavioral properties in relation to the conventional synchronous machine (SM). Next, we introduce the required complementary controls to be combined with the HAC to realize a GFM behavior. The characterization of the analytical operating point and nonlinear energy-based stability analysis of a grid-connected converter under the HAC is presented. Further, we consider various output filter configurations and derive an approximation for the original control proposal. Moreover, we provide details on the integration of GFM HAC into a complex converter control architecture and introduce several variants of the standard HAC. Finally, the performance of GFM HAC is verified by several test scenarios in a C-HiL setup to test its behavior against real-world effect such as noise and delays.

math.OC↗

arXiv2023

On the Guarantees of Minimizing Regret in Receding Horizon

Towards bridging classical optimal control and online learning, regret minimization has recently been proposed as a control design criterion. This competitive paradigm penalizes the loss relative to the optimal control actions chosen by a clairvoyant policy, and allows tracking the optimal performance in hindsight no matter how disturbances are generated. In this paper, we propose the first receding horizon scheme based on the repeated computation of finite horizon regret-optimal policies, and we establish stability and safety guarantees for the resulting closed-loop system. Our derivations combine novel monotonicity properties of clairvoyant policies with suitable terminal ingredients. We prove that our scheme is recursively feasible, stabilizing, and that it achieves bounded regret relative to the infinite horizon clairvoyant policy. Last, we show that the policy optimization problem can be solved efficiently through convex-concave programming. Our numerical experiments show that minimizing regret can outperform standard receding horizon approaches when the disturbances poorly fit classical design assumptions - even when the finite horizon planning is recomputed less frequently.

eess.SY↗

arXiv2023

Regularization for distributionally robust state estimation and prediction

The increasing availability of sensing techniques provides a great opportunity for engineers to design state estimation methods, which are optimal for the system under observation and the observed noise patterns. However, these patterns often do not fulfill the assumptions of existing methods. We provide a direct method using samples of the noise to create a moving horizon observer for linear time-varying and nonlinear systems, which is optimal under the empirical noise distribution. Moreover, we show how to enhance the observer with distributional robustness properties in order to handle unmodeled components in the noise profile, as well as different noise realizations. We prove that, even though the design of distributionally robust estimators is a complex minmax problem over an infinite-dimensional space, it can be transformed into a regularized linear program using a system level synthesis approach. Numerical experiments with the Van der Pol oscillator show the benefits of not only using empirical samples of the noise to design the state estimator, but also of adding distributional robustness. We show that our method can significantly outperform state-of-the-art approaches under challenging noise distributions, including multi-modal and deterministic components.

eess.SY↗

arXiv2023

A self-contained karma economy for the dynamic allocation of common resources

This paper presents karma mechanisms, a novel approach to the repeated allocation of a scarce resource among competing agents over an infinite time. Examples include deciding which ride hailing trip requests to serve during peak demand, granting the right of way in intersections or lane mergers, or admitting internet content to a regulated fast channel. We study a simplified yet insightful formulation of these problems where at every instant two agents from a large population get randomly matched to compete over the resource. The intuitive interpretation of a karma mechanism is "If I give in now, I will be rewarded in the future." Agents compete in an auction-like setting where they bid units of karma, which circulates directly among them and is self-contained in the system. We demonstrate that this allows a society of self-interested agents to achieve high levels of efficiency without resorting to a (possibly problematic) monetary pricing of the resource. We model karma mechanisms as dynamic population games and guarantee the existence of a stationary Nash equilibrium. We then analyze the performance at the stationary Nash equilibrium numerically. For the case of homogeneous agents, we compare different mechanism design choices, showing that it is possible to achieve an efficient and ex-post fair allocation when the agents are future aware. Finally, we test the robustness against agent heterogeneity and propose remedies to some of the observed phenomena via karma redistribution.

econ.TH↗

arXiv2023

SOS Construction of Compatible Control Lyapunov and Barrier Functions

We propose a novel approach to certify closed-loop stability and safety of a constrained polynomial system based on the combination of Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs). For polynomial systems that are affine in the control input, both classes of functions can be constructed via Sum Of Squares (SOS) programming. Using two versions of the Positivstellensatz we derive an SOS formulation seeking a rational controller that - if feasible - results in compatible CLF and multiple CBFs.

math.OC↗