arXiv subjects

Keyou You

Publications and source records attributed to Keyou You.

At least 19 recordsLinked to original sources

arXiv2026

What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data Selection

On-Policy Distillation (OPD) has emerged as a widely adopted post-training paradigm for enhancing large language models in reasoning domains. However, the data-centric mechanisms in OPD remain relatively underexplored. This paper presents a empirical study of data efficiency and data selection in OPD. We begin by investigating an extreme setting: training OPD on only one example, namely 1-shot OPD. Surprisingly, we find that 1-shot OPD is consistently effective across all sampled training examples and harder examples often yield superior performance gain. We next investigate what actually drives the student model's improvement in the training data. Our analysis reveals that the improvement is not driven by high token entropy, but the longer CoT paths which hard problems naturally generate. Training on longer CoT can help maintain closer alignment with the teacher over a long reasoning horizon, and learn critical thinking patterns usually missing in short CoTs, such as reflection (e.g., ``Alternatively''). Based on these insights, we propose a simple data selection method that selects only hard examples for training, where even ``unsolvable'' examples that completely exceed the teacher's capability can be successfully used. Our experiments conducted on four models ranging from 1.5B to 7B show that training the student model on only 8 selected hard examples matches the performance of the 17K dataset baseline.

cs.AI↗

arXiv2026

Robust Koopman MPC with Sets Updates for Time Delayed Systems

Koopman operators have shown significant potential in designing linear model predictive control (MPC) schemes for nonlinear systems on a lifted observable space. Recent advances have tackled the robust Koopman MPC design issue in the presence of modeling errors, relying on the prior estimation of the modeling uncertainty set. However, deriving a robust positively invariant set using a precalculated uncertainty set can be conservative because the uncertainty set bound is time-varying and dependent on the state and control. Additionally, no existing Koopman MPC design has addressed the closed-loop robustness challenge for nonlinear time delayed systems. Thereby, this article presents a robust adaptive Koopman MPC approach with online updates of uncertainty sets for a class of nonlinear time delayed systems. The unknown nonlinear time delayed system is first modeled in a data-driven manner to derive a lifted time delayed Koopman model in the feature space. By analyzing fundamental properties such as controllability and observability, a robust tube-based MPC algorithm is designed for the time delayed Koopman model. The robust adaptive Koopman MPC algorithm with online updates of the uncertainty sets is then presented to reduce conservatism. Closed-loop robustness under exogenous disturbances and asymptotic convergence in the nominal scenario are proven. Finally, numerical examples verify the effectiveness of the proposed approach.

math.OC↗

arXiv2026

APX-Hardness of Computing Lipschitz Constants for Multi-Parametric Quadratic Programs

Computing the Lipschitz constant of the solution map of a multi-parametric quadratic program is important for the analysis of optimization-based control. This problem is governed by three factors: the parameter dimension, the number of decision variables, and the number of constraints. While empirical evidence has long suggested exponential complexity, a rigorous complexity-theoretic proof has been lacking. In this paper, we fill this gap by proving that this problem is not only NP-hard but also APX-hard. Furthermore, we reveal that: (a) the problem becomes polynomial-time solvable when the number of constraints or decision variables is fixed; and (b) both NP-hardness and APX-hardness persist even in the scalar parameter case. These results confirm that the complexity stems from the number of constraints and variables, rather than the parameter dimension. Numerical experiments further validate these theoretical findings.

eess.SY↗

arXiv2026

Direct Data-Driven Linear Quadratic Tracking via Policy Optimization

Direct data-driven optimal control provides an elegant end-to-end paradigm, yet its real-time applicability is often hindered by the growing dimensionality of online decision variables. Recent breakthroughs, notably Data-EnablEd Policy Optimization (DeePO), overcome this bottleneck for the Linear Quadratic Regulator (LQR) through sample-covariance parameterization; however, extending this paradigm to Linear Quadratic Tracking (LQT) poses a fundamental challenge. The core difficulty stems from the intricate coupling between time-varying references and the feedback-feedforward policy structure, which prevents a direct application of constant-dimension parameterization. We first introduce a reference-decoupled reformulation of LQT that naturally accommodates the covariance parameterization, guaranteeing a fixed dimension of decision variables independent of data horizon. This formulation is proven to be exactly equivalent to the indirect certainty-equivalence LQT solution. Leveraging this characterization, we develop offline and online DeePO algorithms. Theoretically, we prove global linear convergence for the offline algorithm using local gradient dominance and smoothness, and show that in the online setting the optimality gap decays linearly up to a bias term that scales inversely with the signal-to-noise ratio (SNR). Numerical simulations varify the theoretical results and illustrate the superior tracking performance of the proposed method.

eess.SY↗

arXiv2026

LLM4Branch: Large Language Model for Discovering Efficient Branching Policies of Integer Programs

Efficient branching policies are essential for accelerating Mixed Integer Linear Programming (MILP) solvers. Their design has long relied on hand-crafted heuristics, and now machine learning has emerged as a promising paradigm to automate this process. However, existing learning-based methods are often hindered by their dependence on expensive expert demonstrations and the gap between training objectives and the solver's end-to-end performance. In this work, we propose LLM4Branch, a novel framework that leverages Large Language Models (LLMs) to automate the discovery of efficient branching policies. Specifically, the discovered policy is an executable program with a program skeleton generated by the LLM and a parameter vector, which is optimized via a zeroth-order method over a few instances with their end-to-end performance feedback. Extensive experiments on standard MILP benchmarks demonstrate that LLM4Branch establishes a new state-of-the-art among CPU-based methods and achieves performance competitive with advanced GPU-based models. Codes are available at https://github.com/hzn18/LLM4Branch.

cs.AI↗

arXiv2026

Adaptive Polyak Stepsize with Level-value Adjustment for Distributed Optimization

Stepsize selection remains a critical challenge in the practical implementation of distributed optimization. Existing distributed algorithms often rely on restrictive prior knowledge of global objective functions, such as Lipschitz constants. While centralized Polyak stepsizes have recently gained attention for their parameter-free adaptability and fast convergence. However, their extension to distributed settings is hindered by the requirement for local function values at the global optimum, which are typically unavailable to individual agents. To bridge this gap, we design a novel distributed adaptive Polyak stepsize algorithm with level-value adjustment (DPS-LA), where each agent only needs to solve a computationally efficient linear feasibility problem, thereby eliminating the dependency on global optimal values. Theoretical analysis proves that DPS-LA guarantees network consensus and achieves a linear speedup convergence rate of $\mathcal{O}(1/\sqrt{nT})$. Numerical results confirm the efficiency of the proposed algorithm.

math.OC↗

arXiv2026

Loopless Proximal Riemannian Gradient EXTRA for Distributed Optimization on Compact Manifolds

Distributed optimization has gained substantial interest in recent years due to its wide applications in machine learning. However, most of existing algorithms are designed for Euclidean spaces, leaving composite optimization on Riemannian manifolds largely unexplored. To bridge this gap, we propose the proximal Riemannian gradient EXTRA algorithm (PR-EXTRA) to solve distributed composite optimization problem with nonsmooth regularizer over compact manifolds. In each iteration, PR-EXTRA requires only a single round communication, coupled with local gradient evaluations and proximal mappings. Furthermore, a manifold projection operator is integrated to ensure the feasibility of all iterates throughout the optimization process. Theoretical analysis shows that with a constant stepsize, PR-EXTRA achieves a sublinear convergence rate of $\mathcal{O}(1/K)$ to a stationary point, matching the proximal gradient EXTRA algorithm in Euclidean spaces. Numerical experiments show the effectiveness of the proposed algorithm.

math.OC↗

arXiv2026

Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data

Federated Composite Optimization (FCO) has emerged as a promising framework for training models with structural constraints (e.g., sparsity) in distributed edge networks. However, simultaneously achieving communication efficiency and convergence robustness remains a significant challenge, particularly when dealing with non-smooth regularizers, statistical heterogeneity, and the restrictions of biased compression. To address these issues, we propose FedCEF (Federated Composite Error Feedback), a novel algorithm tailored for non-convex FCO. FedCEF introduces a decoupled proximal update scheme that separates the proximal operator from communication, enabling clients to handle non-smooth terms locally while transmitting compressed information. To mitigate the noise from aggressive quantization and the bias from non-IID data, FedCEF integrates a rigorous error feedback mechanism with control variates. Furthermore, we design a communication-efficient pre-proximal downlink strategy that allows clients to exactly reconstruct global control variables without explicit transmission. We theoretically establish that FedCEF achieves sublinear convergence to a bounded residual error under general non-convexity, which is controllable via the step size and batch size. Extensive experiments on real datasets validate FedCEF maintains competitive model accuracy even under extreme compression ratios (e.g., 1%), significantly reducing the total communication volume compared to uncompressed baselines.

math.OC↗

arXiv2026

Heterogeneous Stochastic Momentum ADMM for Distributed Nonconvex Composite Optimization

This paper investigates the distributed stochastic nonconvex and nonsmooth composite optimization problem. Existing stochastic typically rely on uniform step size strictly bounded by global network parameters, such as the maximum node degree or spectral radius. This dependency creates a severe performance bottleneck, particularly in heterogeneous network topologies where the step size must be conservatively reduced to ensure stability. To overcome this limitation, we propose a novel Heterogeneous Stochastic Momentum Alternating Direction Method of Multipliers (HSM-ADMM). By integrating a recursive momentum estimator (STORM), HSM-ADMM achieves the optimal oracle complexity of $\mathcal{O}(\epsilon^{-1.5})$ to reach an $\epsilon$-stationary point, utilizing a strictly single-loop structure and an $\mathcal{O}(1)$ mini-batch size. The core innovation lies in a node-specific adaptive step-size strategy, which scales the proximal term according to local degree information. We theoretically demonstrate this design completely decouples the algorithmic stability from global network properties, enabling robust and accelerated convergence across arbitrary connected topologies without requiring any global structural knowledge. Furthermore, HSM-ADMM requires transmitting only a single primal variable per iteration, significantly reducing communication bandwidth compared to state-of-the-art gradient tracking algorithms. Extensive numerical experiments on distributed nonconvex learning tasks validate the superior efficiency of the proposed HSM-ADMM algorithm.

math.OC↗

arXiv2026

A Unified Hybrid Control Architecture for Multi-DOF Robotic Manipulators

Multi-degree-of-freedom (DOF) robotic manipulators exhibit strongly nonlinear, high-dimensional, and coupled dynamics, posing significant challenges for controller design. To address these issues, this work proposes a unified hybrid control architecture that integrates model predictive control (MPC) with feedback regulation, together with a stability analysis of the proposed scheme. The proposed approach mitigates the optimization difficulty associated with high-dimensional nonlinear systems and enhances overall control performance. Furthermore, a hardware implementation scheme based on machine learning (ML) is proposed to achieve high computational efficiency while maintaining control accuracy. Finally, simulation and hardware experiments under external disturbances validate the proposed architecture, demonstrating its superior performance, hardware feasibility, and generalization capability for multi-DOF manipulation tasks.

eess.SY↗

arXiv2026

ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees

Recent years have witnessed a resurgence in using ReLU neural networks (NNs) to represent model predictive control (MPC) policies. However, determining the required network complexity to ensure closed-loop performance remains a fundamental open problem. This involves a critical precision-complexity trade-off: undersized networks may fail to capture the MPC policy, while oversized ones may outweigh the benefits of ReLU network approximation. In this work, we propose a projection-based method to enforce hard constraints and establish a state-dependent Lipschitz continuity property for the optimal MPC cost function, which enables sharp convergence analysis of the closed-loop system. For the first time, we derive explicit bounds on ReLU network width and depth for approximating MPC policies with guaranteed closed-loop performance. To further reduce network complexity and enhance closed-loop performance, we propose a non-uniform error framework with a state-aware scaling function to adaptively adjust both the input and output of the ReLU network. Our contributions provide a foundational step toward certifiable ReLU NN-based MPC.

eess.SY↗

arXiv2025

Safety Meets Speed: Accelerated Neural MPC with Safety Guarantees and No Retraining

While Model Predictive Control (MPC) enforces safety via constraints, its real-time execution can exceed embedded compute budgets. We propose a Barrier-integrated Adaptive Neural Model Predictive Control (BAN-MPC) framework that synergizes neural networks' fast computation with MPC's constraint-handling capability. To ensure strict safety, we replace traditional Euclidean distance with Control Barrier Functions (CBFs) for collision avoidance. We integrate an offline-learned neural value function into the optimization objective of a Short-horizon MPC, substantially reducing online computational complexity. Additionally, we use a second neural network to learn the sensitivity of the value function to system parameters, and adaptively adjust the neural value function based on this neural sensitivity when model parameters change, eliminating the need for retraining and reducing offline computation costs. The hardware in-the-loop (HIL) experiments on Jetson Nano show that BAN-MPC solves 200 times faster than traditional MPC, enabling collision-free navigation with control error below 5\% under model parameter variations within 15\%, making it an effective embedded MPC alternative.

cs.RO↗

arXiv2025

Data-driven Internal Model Control for Output Regulation

Output regulation is a fundamental problem in control theory, extensively studied since the 1970s. Traditionally, research has primarily addressed scenarios where the system model is explicitly known, leaving the problem in the absence of a system model less explored. Leveraging the recent advancements in Willems et al.'s fundamental lemma, data-driven control has emerged as a powerful tool for stabilizing unknown systems. This paper tackles the output regulation problem for unknown single and multi-agent systems (MASs) using noisy data. Previous approaches have attempted to solve data-based output regulation equations (OREs), which are inadequate for achieving zero tracking error with noisy data. To circumvent the need for solving data-based OREs, we propose an internal model-based data-driven controller that reformulates the output regulation problem into a stabilization problem. This method is first applied to linear time-invariant (LTI) systems, demonstrating exact solution capabilities, i.e., zero tracking error, through solving a straightforward data-based linear matrix inequality (LMI). Furthermore, we extend our approach to solve the $k$th-order output regulation problem for nonlinear systems. Extensions to both linear and nonlinear MASs are discussed. Finally, numerical tests validate the effectiveness and correctness of the proposed controllers.

eess.SY↗

arXiv2024

Reliably Learn to Trim Multiparametric Quadratic Programs via Constraint Removal

In a wide range of applications, we are required to rapidly solve a sequence of convex multiparametric quadratic programs (mp-QPs) on resource-limited hardwares. This is a nontrivial task and has been an active topic for decades in control and optimization communities. Observe that the main computational cost of existing solution algorithms lies in addressing many linear inequality constraints, though their majority are redundant and removing them will not change the optimal solution. This work learns from the results of previously solved mp-QP(s), based on which we propose novel methods to reliably trim (unsolved) mp-QPs via constraint removal, and the trimmed mp-QPs can be much cheaper to solve. Then, we extend to trim mp-QPs of model predictive control (MPC) whose parameter vectors are sampled from linear systems. Importantly, both online and offline solved mp-QPs can be utilized to adaptively trim mp-QPs in the closed-loop system. We show that the number of linear inequalities in the trimmed mp-QP of MPC decreases to zero in a finite timestep, which also can be reduced by increasing offline computation. Finally, simulations are performed to demonstrate the efficiency of our trimming method in removing redundant constraints.

math.OC↗

arXiv2024

Direct Adaptive Control of Grid-Connected Power Converters via Output-Feedback Data-Enabled Policy Optimization

Power electronic converters are becoming the main components of modern power systems due to the increasing integration of renewable energy sources. However, power converters may become unstable when interacting with the complex and time-varying power grid. In this paper, we propose an adaptive data-driven control method to stabilize power converters by using only online input-output data. Our contributions are threefold. First, we reformulate the output-feedback control problem as a state-feedback linear quadratic regulator (LQR) problem with a controllable non-minimal state, which can be constructed from past input-output signals. Second, we propose a data-enabled policy optimization (DeePO) method for this non-minimal realization to achieve efficient output-feedback adaptive control. Third, we use high-fidelity simulations to verify that the output-feedback DeePO can effectively stabilize grid-connected power converters and quickly adapt to the changes in the power grid.

eess.SY↗

arXiv2024

Linear Convergence of Data-Enabled Policy Optimization for Linear Quadratic Tracking

Data-enabled policy optimization (DeePO) is a newly proposed method to attack the open problem of direct adaptive LQR. In this work, we extend the DeePO framework to the linear quadratic tracking (LQT) with offline data. By introducing a covariance parameterization of the LQT policy, we derive a direct data-driven formulation of the LQT problem. Then, we use gradient descent method to iteratively update the parameterized policy to find an optimal LQT policy. Moreover, by revealing the connection between DeePO and model-based policy optimization, we prove the linear convergence of the DeePO iteration. Finally, a numerical experiment is given to validate the convergence results. We hope our work paves the way to direct adaptive LQT with online closed-loop data.

eess.SY↗

arXiv2024

Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator

Learning policies in an asynchronous parallel way is essential to the numerous successes of RL for solving large-scale problems. However, their convergence performance is still not rigorously evaluated. To this end, we adopt the asynchronous parallel zero-order policy gradient (AZOPG) method to solve the continuous-time linear quadratic regulation problem. Specifically, as in the celebrated A3C algorithm, there are multiple parallel workers to asynchronously estimate PGs which are then sent to a central master for policy updates. Via quantifying its convergence rate of policy iterations, we show the linear speedup property of the AZOPG, both in theory and simulation, which clearly reveals the advantages of using parallel workers for learning policies.

math.OC↗

arXiv2024

Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence

In safety-critical applications, reinforcement learning (RL) needs to consider safety constraints. However, theoretical understandings of constrained RL for continuous control are largely absent. As a case study, this paper presents a cost-constrained LQR formulation, where a number of LQR costs with user-defined penalty matrices are subject to constraints. To solve it, we propose a policy gradient primal-dual method to find an optimal state feedback gain. Despite the non-convexity of the cost-constrained LQR problem, we provide a constructive proof for strong duality and a geometric interpretation of an optimal multiplier set. By proving that the concave dual function is Lipschitz smooth, we further provide convergence guarantees for the PG primal-dual method. Finally, we perform simulations to validate our theoretical findings.

math.OC↗