arXiv subjects

Florian Dörfler

Publications and source records attributed to Florian Dörfler.

At least 109 records · Page 6Linked to original sources

arXiv2025

Mixed Small Gain and Phase Theorem: A new view using Scale Relative Graphs

We introduce a novel approach to feedback stability analysis for linear time-invariant (LTI) systems, overcoming the limitations of the sectoriality assumption in the small phase theorem. While phase analysis for single-input single-output (SISO) systems is well-established, multi-input multi-output (MIMO) systems lack a comprehensive phase analysis until recent advances introduced with the small-phase theorem. A limitation of the small-phase theorem is the sectorial condition, which states that an operator's eigenvalues must lie within a specified angle sector of the complex plane. We propose a framework based on Scaled Relative Graphs (SRGs) to remove this assumption. We derive two main results: a graphical set-based stability condition using SRGs and a small-phase theorem with no sectorial assumption. These results broaden the scope of phase analysis and feedback stability for MIMO systems.

math.OC↗

arXiv2025

Safety Filter for Limiting the Current of Grid-Forming Matrix Modular Multilevel Converters

Grid-forming (GFM) converters face significant challenges in limiting current during transient grid events while preserving their grid-forming behavior. This paper offers an elegant solution to the problem with a priori guarantees, presenting a safety filter approach based on Control Barrier Functions (CBFs) to enforce current constraints with minimal deviation from the nominal voltage reference. The safety filter is implemented as a Quadratic Program, enabling real-time computation of safe voltage adjustments that ensure smooth transitions and maintain the GFM behavior during nominal operation. To provide formal safety certificate, the CBF is synthesized offline using a Sum-of-Squares optimization framework, ensuring that the converter remains within its allowable operating limits under all conditions. Additionally, a Control Lyapunov Function is incorporated to facilitate a smooth return to the nominal operating region following grid events. The proposed method is modular and can be integrated into many of the GFM control architectures, as demonstrated with two different GFM implementations. High-fidelity simulations conducted with an enhanced matrix modular multilevel converter connected to both high-inertia and low-inertia grid scenarios validate the effectiveness of the safety filter, showing that it successfully limits current during faults, preserves GFM behavior, and ensures a seamless recovery to nominal operation.

eess.SY↗

arXiv2025

Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics

Distribution shifts have long been regarded as troublesome external forces that a decision-maker should either counteract or conform to. An intriguing feedback phenomenon termed decision dependence arises when the deployed decision affects the environment and alters the data-generating distribution. In the realm of performative prediction, this is encoded by distribution maps parameterized by decisions due to strategic behaviors. In contrast, we formalize an endogenous distribution shift as a feedback process featuring nonlinear dynamics that couple the evolving distribution with the decision. Stochastic optimization in this dynamic regime provides a fertile ground to examine the various roles played by dynamics in the composite problem structure. To this end, we develop an online algorithm that achieves optimal decision-making by both adapting to and shaping the dynamic distribution. Throughout the paper, we adopt a distributional perspective and demonstrate how this view facilitates characterizations of distribution dynamics and the optimality and generalization performance of the proposed algorithm. We showcase the theoretical results in an opinion dynamics context, where an opportunistic party maximizes the affinity of a dynamic polarized population, and in a recommender system scenario, featuring performance optimization with discrete distributions in the probability simplex.

math.OC↗

arXiv2025

Regularization for Covariance Parameterization of Direct Data-Driven LQR Control

As the benchmark of data-driven control methods, the linear quadratic regulator (LQR) problem has gained significant attention. A growing trend is direct LQR design, which finds the optimal LQR gain directly from raw data and bypassing system identification. To achieve this, our previous work develops a direct LQR formulation parameterized by sample covariance. In this paper, we propose a regularization method for the covariance-parameterized LQR. We show that the regularizer accounts for the uncertainty in both the steady-state covariance matrix corresponding to closed-loop stability, and the LQR cost function corresponding to averaged control performance. With a positive or negative coefficient, the regularizer can be interpreted as promoting either exploitation or exploration, which are well-known trade-offs in reinforcement learning. In simulations, we observe that our covariance-parameterized LQR with regularization can significantly outperform the certainty-equivalence LQR in terms of both the optimality gap and the robust closed-loop stability.

eess.SY↗

arXiv2025

First-order Conditions for Optimization in the Wasserstein Space

We study first-order optimality conditions for constrained optimization in the Wasserstein space, whereby one seeks to minimize a real-valued function over the space of probability measures endowed with the Wasserstein distance. Our analysis combines recent insights on the geometry and the differential structure of the Wasserstein space with more classical calculus of variations. We show that simple rationales such as "setting the derivative to zero" and "gradients are aligned at optimality" carry over to the Wasserstein space. We deploy our tools to study and solve optimization problems in the setting of distributionally robust optimization and statistical inference. The generality of our methodology allows us to naturally deal with functionals, such as mean-variance, Kullback-Leibler divergence, and Wasserstein distance, which are traditionally difficult to study in a unified framework.

math.OC↗

arXiv2025

An Adaptive Data-Enabled Policy Optimization Approach for Autonomous Bicycle Control

This paper presents a unified control framework that integrates a Feedback Linearization (FL) controller in the inner loop with an adaptive Data-Enabled Policy Optimization (DeePO) controller in the outer loop to balance an autonomous bicycle. While the FL controller stabilizes and partially linearizes the inherently unstable and nonlinear system, its performance is compromised by unmodeled dynamics and time-varying characteristics. To overcome these limitations, the DeePO controller is introduced to enhance adaptability and robustness. The initial control policy of DeePO is obtained from a finite set of offline, persistently exciting input and state data. To improve stability and compensate for system nonlinearities and disturbances, a robustness-promoting regularizer refines the initial policy, while the adaptive section of the DeePO framework is enhanced with a forgetting factor to improve adaptation to time-varying dynamics. The proposed DeePO+FL approach is evaluated through simulations and real-world experiments on an instrumented autonomous bicycle. Results demonstrate its superiority over the FL-only approach, achieving more precise tracking of the reference lean angle and lean rate.

eess.SY↗

arXiv2025

Optimizing Social Network Interventions via Hypergradient-Based Recommender System Design

Although social networks have expanded the range of ideas and information accessible to users, they are also criticized for amplifying the polarization of user opinions. Given the inherent complexity of these phenomena, existing approaches to counteract these effects typically rely on handcrafted algorithms and heuristics. We propose an elegant solution: we act on the network weights that model user interactions on social networks (e.g., frequency of communication), to optimize a performance metric (e.g., polarization reduction), while users' opinions follow the classical Friedkin-Johnsen model. Our formulation gives rise to a challenging large-scale optimization problem with non-convex constraints, for which we develop a gradient-based algorithm. Our scheme is simple, scalable, and versatile, as it can readily integrate different, potentially non-convex, objectives. We demonstrate its merit by: (i) rapidly solving complex social network intervention problems with 3 million variables based on the Reddit and DBLP datasets; (ii) significantly outperforming competing approaches in terms of both computation time and disagreement reduction.

cs.SI↗

arXiv2025

NeoRL: Efficient Exploration for Nonepisodic RL

We study the problem of nonepisodic reinforcement learning (RL) for nonlinear dynamical systems, where the system dynamics are unknown and the RL agent has to learn from a single trajectory, i.e., without resets. We propose Nonepisodic Optimistic RL (NeoRL), an approach based on the principle of optimism in the face of uncertainty. NeoRL uses well-calibrated probabilistic models and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics. Under continuity and bounded energy assumptions on the system, we provide a first-of-its-kind regret bound of $O(Γ_T \sqrt{T})$ for general nonlinear systems with Gaussian process dynamics. We compare NeoRL to other baselines on several deep RL environments and empirically demonstrate that NeoRL achieves the optimal average cost while incurring the least regret.

cs.LG↗

arXiv2025

To Spend or to Gain: Online Learning in Repeated Karma Auctions

Recent years have seen a surge of artificial currency-based mechanisms in contexts where monetary instruments are deemed unfair or inappropriate, e.g., in allocating food donations to food banks, course seats to students, and, more recently, even for traffic congestion management. Yet the applicability of these mechanisms remains limited in repeated auction settings, as it is challenging for users to learn how to bid an artificial currency that has no value outside the auctions. Indeed, users must jointly learn the value of the currency in addition to how to spend it optimally. Moreover, in the prominent class of karma mechanisms, in which artificial karma payments are redistributed to users at each time step, users do not only spend karma to obtain public resources but also gain karma for yielding them. For this novel class of karma auctions, we propose an adaptive karma pacing strategy that learns to bid optimally, and show that this strategy a) is asymptotically optimal for a single user bidding against competing bids drawn from a stationary distribution; b) leads to convergent learning dynamics when all users adopt it; and c) constitutes an approximate Nash equilibrium as the number of users grows. Our results require a novel analysis in comparison to adaptive pacing strategies in monetary auctions, since we depart from the classical assumption that the currency has known value outside the auctions, and consider that the currency is both spent and gained through the redistribution of payments.

cs.GT↗

arXiv2024

To Travel Quickly or to Park Conveniently: Coupled Resource Allocations with Multi-Karma Economies

The large-scale allocation of public resources (e.g., transportation, energy) is among the core challenges of future Cyber-Physical-Human Systems (CPHS). In order to guarantee that these systems are efficient and fair, recent works have investigated non-monetary resource allocation schemes, including schemes that employ karma. Karma is a non-tradable token that flows from users gaining resources to users yielding resources. Thus far karma-based solutions considered the allocation of a single public resource, however, modern CPHS are complex as they involve the allocation of multiple coupled resources. For example, a user might want to trade-off fast travel on highways for convenient parking in the city center, and different users could have heterogeneous preferences for such coupled resources. In this paper, we explore how to optimally combine multiple karma economies for coupled resource allocations, using two mechanism-design instruments: (non-uniform) karma redistribution; and (non-unit) exchange rates. We first extend the existing Dynamic Population Game (DPG) model that predicts the Stationary Nash Equilibrium (SNE) of the multi-karma economies. Then, in a numerical case study, we demonstrate that the design of redistribution significantly affects the coupled resource allocations, while non-unit exchange rates play a minor role. To assess the allocation outcomes under user heterogeneity, we adopt Nash welfare as our social welfare function, since it makes no interpersonal comparisons and it is axiomatically rooted in social choice theory. Our findings suggest that the simplest mechanism design, that is, uniform redistribution with unit exchange rates, also attains maximum social welfare.

cs.GT↗

arXiv2024

Cross-Forming Control and Fault Current Limiting for Grid-Forming Inverters

This article proposes a "cross-forming" control concept for grid-forming inverters operating against grid faults. Cross-forming refers to voltage angle forming and current magnitude forming. It differs from classical grid-forming and grid-following paradigms that feature voltage magnitude-and-angle forming and voltage magnitude-and-angle following (or current magnitude-and-angle forming), respectively. The cross-forming concept addresses the need for inverters to remain grid-forming (particularly voltage angle forming, as required by grid codes) while managing fault current limitation. Simple and feasible cross-forming control implementations are proposed, enabling inverters to quickly limit fault currents to a prescribed level while preserving voltage angle forming for grid-forming synchronization and providing dynamic ancillary services, during symmetrical or asymmetrical fault ride-through. Moreover, the cross-forming control yields an equivalent system featuring a constant virtual impedance and a "normal form" representation, allowing for the extension of previously established transient stability results to include scenarios involving current saturation. Simulations and experiments validate the efficacy of the proposed cross-forming control implementations.

eess.SY↗

arXiv2024

Data-Driven Structured Robust Control of Linear Systems

Static structured control refers to the task of designing a state-feedback controller such that the control gain satisfies a subspace constraint. Structured control has applications in control of communication-inhibited dynamical systems, such as systems in networked environments. This work performs $H_2$-suboptimal regulation under a common structured state-feedback controller for a class of data-consistent plants. The certification of $H_2$-performance is attained through a combination of standard $H_2$ LMIs, convex sufficient conditions for structured control, and a matrix S-lemma for set-membership. The resulting convex optimization problems are linear matrix inequalities whose size scales independently of the number of data samples collected. Data-driven structured $H_2$-regulation control is demonstrated on example systems.

math.OC↗

arXiv2024

When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL

Reinforcement learning (RL) excels in optimizing policies for discrete-time Markov decision processes (MDP). However, various systems are inherently continuous in time, making discrete-time MDPs an inexact modeling choice. In many applications, such as greenhouse control or medical treatments, each interaction (measurement or switching of action) involves manual intervention and thus is inherently costly. Therefore, we generally prefer a time-adaptive approach with fewer interactions with the system. In this work, we formalize an RL framework, Time-adaptive Control & Sensing (TaCoS), that tackles this challenge by optimizing over policies that besides control predict the duration of its application. Our formulation results in an extended MDP that any standard RL algorithm can solve. We demonstrate that state-of-the-art RL algorithms trained on TaCoS drastically reduce the interaction amount over their discrete-time counterpart while retaining the same or improved performance, and exhibiting robustness over discretization frequency. Finally, we propose OTaCoS, an efficient model-based algorithm for our setting. We show that OTaCoS enjoys sublinear regret for systems with sufficiently smooth dynamics and empirically results in further sample-efficiency gains.

cs.LG↗

arXiv2024

Learning diffusion at lightspeed

Diffusion regulates numerous natural processes and the dynamics of many successful generative models. Existing models to learn the diffusion terms from observational data rely on complex bilevel optimization problems and model only the drift of the system. We propose a new simple model, JKOnet*, which bypasses the complexity of existing architectures while presenting significantly enhanced representational capabilities: JKOnet* recovers the potential, interaction, and internal energy components of the underlying diffusion process. JKOnet* minimizes a simple quadratic loss and outperforms other baselines in terms of sample efficiency, computational complexity, and accuracy. Additionally, JKOnet* provides a closed-form optimal solution for linearly parametrized functionals, and, when applied to predict the evolution of cellular processes from real-world data, it achieves state-of-the-art accuracy at a fraction of the computational cost of all existing methods. Our methodology is based on the interpretation of diffusion processes as energy-minimizing trajectories in the probability space via the so-called JKO scheme, which we study via its first-order optimality conditions.

cs.LG↗

arXiv2024

Grid-Forming Control of Modular Dynamic Virtual Power Plants

This article explores a flexible and coordinated control design for an aggregation of heterogeneous distributed energy resources (DERs) in a dynamic virtual power plant (DVPP). The control design aims to provide a desired aggregate grid-forming (GFM) response based on the coordination of power contributions between different DERs. Compared to existing DVPP designs with an AC-coupled AC-output configuration, a more generic modular DVPP design is proposed in this article, which comprises four types of basic DVPP modules, involving AC- or DC-coupling and AC- or DC-output, adequately accommodating diverse DER integration setups, such as AC, DC, AC/DC hybrid microgrids and renewable power plants. The control design is first developed for the four basic modules by the aggregation of DERs and the disaggregation of the control objectives, and then extended to modular DVPPs through a systematic top-down approach. The control performance is comprehensively validated through simulation. The modular DVPP design offers scalable and standardizable advanced grid interfaces (AGIs) for building and operating AC/DC hybrid power grids.

eess.SY↗

arXiv2024

Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQR

Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal non-asymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper-bounded by two terms signifying a sublinear decrease in time $\mathcal{O}(1/\sqrt{T})$ plus a bias scaling inversely with signal-to-noise ratio (SNR), which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method.

math.OC↗

arXiv2024

Network-aware Recommender System via Online Feedback Optimization

Personalized content on social platforms can exacerbate negative phenomena such as polarization, partly due to the feedback interactions between recommendations and the users. In this paper, we present a control-theoretic recommender system that explicitly accounts for this feedback loop to mitigate polarization. Our approach extends online feedback optimization - a control paradigm for steady-state optimization of dynamical systems - to develop a recommender system that trades off users engagement and polarization reduction, while relying solely on online click data. We establish theoretical guarantees for optimality and stability of the proposed design and validate its effectiveness via numerical experiments with a user population governed by Friedkin-Johnsen dynamics. Our results show these "network-aware" recommendations can significantly reduce polarization while maintaining high levels of user engagement.

eess.SY↗

arXiv2024

Decoupling parameter variation from noise: Biquadratic Lyapunov forms in data-driven LPV control

A promising step from linear towards nonlinear data-driven control is via the design of controllers for linear parameter-varying (LPV) systems, which are linear systems whose parameters are varying along a measurable scheduling signal. However, the interplay between uncertainty arising from corrupted data and the parameter-varying nature of these systems impacts the stability analysis and limits the generalization of well-understood data-driven methods available for linear time-invariant systems. In this work, we decouple this interplay using a recently developed variant of the Fundamental Lemma for LPV systems and the concept of data-informativity, in combination with biquadratic Lyapunov forms. Together, these allow us to develop novel linear matrix inequality conditions for the existence of scheduling-dependent Lyapunov functions, incorporating the intrinsic nonlinearity. Appealingly, these results are stated purely in terms of the collected data and bounds on the noise, and they are computationally favorable to check.

eess.SY↗