arXiv subjects

Yuanyuan Shi

Publications and source records attributed to Yuanyuan Shi.

At least 19 recordsLinked to original sources

arXiv2026

Recursive Self-Improvement LLM Agents for Inverter Dynamic Model Identification

This is a position paper. We demonstrate that recursive self-improvement (RSI) large language model (LLM) agents are a natural search engine for dynamic model identification of inverter-based resources (IBRs) whose internal controls are often proprietary and hidden from grid operators. White-box models provide physical transparency but require vendor disclosure; black-box models avoid this requirement but sacrifice interpretability; and existing grey-box approaches, including sparse and symbolic regression, are poorly suited to discovering feedback control architectures or incorporating control-engineering priors. Our position is that this gap can be alleviated by (1) restricting the search space to a typed vocabulary of standard control modules, including PI controllers, phase-locked loops (PLLs), low-pass filters, etc., composed under block-diagram grammar rules, and (2) using an RSI LLM agent to perform program search over candidate block-diagram models, guided by measured frequency-domain admittance data at the point of common coupling (PCC), while fitting the free parameters of each candidate by nonlinear least squares. We instantiate this position by adapting ThetaEvolve, an open-source program-evolution framework supporting in-context evolution and test-time learning, to inverter model discovery. In a proof-of-concept study on a grid-following (GFL) inverter benchmark, the RSI loop reduces the normalized root mean square error (NRMSE) of a naive open-loop model from 0.470 to 0.0435 and identifies a 15-module closed-loop structure that closely resembles the hidden ground-truth GFL controller.

eess.SY↗

arXiv2026

Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL

Reinforcement learning (RL) has emerged as a powerful approach for improving reasoning in language and vision-language models, yet its strongest successes still depend heavily on ground-truth supervision (e.g., verifiable reward). Such annotations are costly to obtain and become increasingly scarce as reasoning capabilities advance beyond what humans can reliably evaluate. Self-rewarding RL reduces this dependence by enabling models to derive reward signals from their own completions. However, training solely on self-generated feedback can reinforce existing biases and suboptimal behaviors, reduce response diversity, and ultimately lead to homogenized responses and training collapse. In this work, we show that unsupervised reasoning can emerge through cooperative multi-agent training. We introduce Co-RL, a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from their peers. We further show that increasing cohort diversity, through heterogeneous model families, sizes, and rephrased training samples, reduces the correlated errors that drive self-reinforcing feedback loops. This diversity consistently improves reasoning performance, maintains behavioral diversity, and mitigates training collapse. Across text-only and multimodal domains, Co-RL consistently outperforms the base models and prior label-free approaches, while matching or surpassing supervised methods, without access to any ground-truth labels. Concretely, Co-RL yields average gains of 3.0-8.6% across seven text-only benchmarks for LLMs and 2.3-7.2% across four multimodal benchmarks for VLMs. Code is available at https://github.com/DrStranded/Co-RL.

cs.LG↗

arXiv2026

Power Estimation and Optimal Work-Charging Scheduling of Construction Electric Vehicles via Mobile Charging Stations

Construction electric vehicles (CEVs) are a promising clean alternative to diesel-powered construction equipment, but their adoption is constrained by sparse onsite charging infrastructure, limited CEV mobility, and insufficient understanding of their power consumption. We address these gaps through a field-data-driven framework coupling CEV power estimation with mobile-charging-aware work scheduling. First, using a real-world construction demonstration at the University of California, San Diego, we develop and validate a per-subactivity power estimation model for a compact electric excavator. Manually labeled video is synchronized with coarse battery state-of-charge (SOC) telematics, and constrained nonnegative least squares is used to recover each subactivity's average power consumption. The model predicts held-out test data within $17\%$ normalized mean absolute error (NMAE), and the accompanying dataset is released publicly. Second, leveraging the subactivity power estimates, we formulate a mixed-integer program that jointly optimizes CEV work and charging schedules together with the location, timing, and charging/discharging of mobile charging stations (MCSs) serving the CEVs. The optimization accounts for energy and demand charges, carbon emissions, unmet work penalties, MCS travel, and the physical and operational constraints of the CEVs and MCSs. Across realistic scenarios drawn from the demonstration, the proposed co-optimization attains the lowest operating cost in every case, being $7$--$96\%$ below the best-performing baseline, while solving most instances to proven optimality within an hour. Dataset and scripts are available at https://github.com/ghosh-avik/CEV-MCS-Power-Estimation-and-Joint-Scheduling.

eess.SY↗

arXiv2026

LLMs and Agentic AI Systems for Smart Grids: A Tutorial on Architectures and Applications

Large language models (LLMs) and agentic AI systems have evolved from natural language tasks to using external tools to plan, retrieve, and act in technical domains. In smart grids, recent work applies agentic schemes to forecasting, optimization, and control, wrapping trusted solvers behind language interfaces and orchestrating multi-step workflows. The literature lacks a unified approach to designing and evaluating such systems. LLMs can produce numerically plausible yet physically infeasible outputs, evaluation protocols vary across tasks, and the boundary between what the model should and should not compute is implicit. This paper presents a solver-grounded design principle: a numerical result is reported only when it originates from a trusted tool and passes explicit verification. We review the building blocks of LLM and agentic AI systems for power systems: prompting strategies and agentic architectures. We instantiate the principle in four case studies: wind power forecasting, EV charging scheduling, power flow analysis, and contingency diagnosis, each comparing an LLM-only baseline against its solver-grounded counterpart on identical data and metrics. EVAgent reproduces the CVXPY optimum while reducing LLM-only unmet energy by 7.5-9.5x, and GridDebugAgent repairs 17/39 contingency cases while reducing total violations by 52.3%. We propose a four-group evaluation framework spanning task utility, solver-grounded correctness, faithfulness and safe failure, and cost and latency. A consistent division of labor emerges: the agentic system reliably orchestrates, retrieves, and explains, while trusted tools compute and a verification gate decides what is reported.

eess.SY↗

arXiv2026

Control-Oriented System Identification: Classical, Learning, and Physics-Informed Approaches

We survey classical, machine learning, and data-driven system identification approaches to learn control-relevant and physics-informed models of dynamical systems. Recently, machine learning approaches have enabled system identification from noisy, high-dimensional, and complex data. However, their utility is limited by their ability to provide provable guarantees on control-relevant properties. Meanwhile, control theory has identified several properties that are useful in analysis and control synthesis, such as dissipativity, monotonicity, energy conservation, and symmetry-preserving structures. We posit that merging system identification with such control-relevant or physics-informed properties can provide useful inductive bias, enhance explainability, enable control synthesis with provable guarantees, and improve sample complexity. We formulate system identification as an optimization problem where control-relevant properties can be enforced through direct parameterization (constraining the model structure to satisfy a desired property by construction), soft constraints (encouraging control-relevant properties through regularization or penalty terms), and hard constraints (imposing control-relevant properties as constraints in the optimization problem). Through this lens, we survey methods to learn physics-informed and control-relevant models spanning classical linear and nonlinear system identification, machine learning approaches, and direct identification through data-driven and behavioral representations. We also provide several expository examples that are accompanied by code and brief tutorials on a public Github repository. We also describe challenging directions for future research, including identification in networked, switched, and time-varying systems, experiment design, and bridging the gaps between data-driven, learning-based, and control-oriented approaches.

eess.SY↗

arXiv2026

RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning

On-policy Reinforcement Learning (RL) remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy, and policy updates must be conservative. In this paper, we revisit actor policy representation as a first-class design choice for on-policy RL. We study discretized categorical actors, which represent each action dimension as a distribution over discrete bins and induce a policy objective analogous to classification cross-entropy loss. Building on architectural advances from supervised learning, we further pair discretized categorical actors with regularized networks, yielding RN-D. Across diverse continuous-control benchmarks, we show that simply replacing the standard Gaussian actor with our proposed actor substantially improves performance, achieving state-of-the-art results within on-policy RL. We release our code at https://github.com/alwaysbyx/RND-RL.

cs.LG↗

arXiv2026

Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning

Scaling critic capacity is a promising direction for improving off-policy reinforcement learning (RL). However, recent work shows that larger critics are prone to overfitting and instability in replay-based bootstrapped training. In this paper, we propose using Low-Rank Adaptation (LoRA) as a structural regularizer for critic learning. Our approach freezes randomly initialized base matrices and optimizes only the corresponding low-rank adapters, thereby constraining critic updates to a low-dimensional subspace. We evaluate our method across different off-policy RL algorithms, including SAC and FastTD3 based on different network architectures. Empirically, LoRA efficiently reduces critic loss during training and improves overall policy performance, achieving the best or competitive results on most tasks. Extensive experiments demonstrate that our low-rank updates provide a simple and effective form of structural regularization for critic learning in off-policy RL.

cs.LG↗

arXiv2026

ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling

Large language models (LLMs) excel at function calling, but inference scaling has been explored mainly for unstructured generation. We propose an inference-scaling framework for structured outputs that combines fine-grained beam search with \textbf{ToolPRM}, a process reward model scoring each intra-call decision (function name and argument filling). We build the first fine-grained intra-call supervision dataset via function masking, rollout collection, and step-level annotation. ToolPRM outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. We further show that structured generation follows ``\textbf{explore more but retain less}'', since early JSON errors are unrecoverable.

cs.AI↗

arXiv2026

Nonvolatile single-ion memory with picosecond switching

The rapid development of artificial intelligence (AI), Internet of Things (IoT), and edge computing applications has posed severe challenges to conventional memory technologies in terms of density, speed, and energy consumption. Herein, a single-ion transport mechanism is proposed to achieve picosecond (ps) switching capability. For monolayer hexagonal boron nitride (h-BN) with single-atom vacancy defects, first-principles calculations reveal that single-ion penetration across the BN plane dominates the resistive switching. The trapping and release of a single ion correspond to different states of the memory device for one bit of information. Experimentally fabricated single-ion memory exhibits nonvolatile resistive switching with ultra-fast switching speed of 20 ps and ultra-low energy consumption of 310 aJ/bit. This high performance is attributed to the extremely short distance for the single ion to travel through. Such devices pave the way for the realization of high-performance nonvolatile memory with ultra-fast speed, ultra-low energy consumption, and high storage density, that is called the "Unified Memory" long desired by the whole industry.

physics.app-ph↗

arXiv2026

Predictor-Based Output-Feedback Control of Linear Systems with Time-Varying Input and Measurement Delays via Neural-Approximated Prediction Horizons

Due to simplicity and strong stability guarantees, predictor feedback methods have stood as a popular approach for time delay systems since the 1950s. For time-varying delays, however, implementation requires computing a prediction horizon defined by the inverse of the delay function, which is rarely available in closed form and must be approximated. In this work, we formulate the inverse delay mapping as an operator learning problem and study predictor feedback under approximation of the prediction horizon. We propose two approaches: (i) a numerical method based on time integration of an equivalent ODE, and (ii) a data-driven method using neural operators to learn the inverse mapping. We show that both approaches achieve arbitrary approximation accuracy over compact sets, with complementary trade-offs in computational cost and scalability. Building on these approximations, we then develop an output-feedback predictor design for systems with delays in both the input and the measurement. We prove that the resulting closed-loop system is globally exponentially stable when the prediction horizon is approximated with sufficiently small error. Lastly, numerical experiments validate the proposed methods and illustrate their trade-offs between accuracy and computational efficiency.

eess.SY↗

arXiv2026

Sampling-Horizon Neural Operator Predictors for Nonlinear Control under Delayed Inputs

Modern control systems frequently operate under input delays and sampled state measurements. A common delay-compensation strategy is predictor feedback; however, practical implementations require solving an implicit ODE online, resulting in intractable computational cost. Moreover, predictor formulations typically assume continuously available state measurements, whereas in practice measurements may be sampled, irregular, or temporarily missing due to hardware faults. In this work, we develop two neural-operator predictor-feedback designs for nonlinear systems with delayed inputs and sampled measurements. In the first design, we introduce a sampling-horizon prediction operator that maps the current measurement and input history to the predicted state trajectory over the next sampling interval. In the second design, the neural operator approximates only the delay-compensating predictor, which is then composed with the closed-loop flow between measurements. The first approach requires uniform sampling but yields residual bounds that scale directly with the operator approximation error. In contrast, the second accommodates non-uniform, but bounded sampling schedules at the cost of amplified approximation error, revealing a practical tradeoff between sampling flexibility and approximation sensitivity for the control engineer. For both schemes, we establish semi-global practical stability with explicit neural operator error-dependent bounds. Numerical experiments on a 6-link nonlinear robotic manipulator demonstrate accurate tracking and substantial computational speedup of 25$\times$ over a baseline approach.

eess.SY↗

arXiv2026

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts

Recent studies have shown that combining parameter-efficient fine-tuning (PEFT) with mixture-of-experts (MoE) is an effective strategy for adapting large language models (LLMs) to the downstream tasks. However, most existing approaches rely on conventional TopK routing, which requires careful hyperparameter tuning and assigns a fixed number of experts to each token. In this work, we propose LD-MoLE, a Learnable Dynamic routing mechanism for Mixture of LoRA Experts that enables adaptive, token-dependent, and layer-wise expert allocation. Our method replaces the non-differentiable TopK selection with a differentiable routing function and a closed-form solution. Moreover, our design allows the model to adaptively determine the number of experts to activate for each token at different layers. In addition, we introduce an analytical sparsity control objective to regularize the number of activated experts. Extensive experiments on the Qwen3-1.7B and Llama-3.2-3B models show that LD-MoLE achieves the highest average scores compared to state-of-the-art baselines, across a diverse set of benchmarks. Our method not only achieves superior performance, but also demonstrates the ability to learn token-dependent and layer-wise expert allocation.

cs.CL↗

arXiv2026

Efficient Policy Adaptation for Voltage Control Under Unknown Topology Changes

Reinforcement learning (RL) has shown great potential for designing voltage control policies, but their performance often degrades under changing system conditions such as topology reconfigurations and load variations. We introduce a topology-aware online policy optimization framework that leverages data-driven estimation of voltage-reactive power sensitivities to achieve efficient policy adaptation. Exploiting the sparsity of topology-switching events, where only a few lines change at a time, our method efficiently detects topology changes and identifies the affected lines and parameters, enabling fast and accurate sensitivity updates without recomputing the full sensitivity matrix. The estimated sensitivity is subsequently used for online policy optimization of a pre-trained neural-network-based RL controller. Simulations on both the IEEE 13-bus and SCE 56-bus systems demonstrate over 90 percent line identification accuracy, using only 15 data points. The proposed method also significantly improves voltage regulation performance compared with non-adaptive policies and adaptive policies that rely on regression-based online optimization methods for sensitivity estimation.

eess.SY↗

arXiv2026

Mobile charges in MoS2/high-k oxide transistors: from abnormal instabilities to memory-like dynamics

MoS$_2$ field-effect transistors (FETs) with high-\textit{k} oxides currently lag behind silicon standards in bias and temperature stability due to ubiquitous border oxide traps that cause clockwise (CW) hysteresis in gate transfer characteristics. While suppressing this effect is typically mandatory for logic FETs, here we explore an alternative strategy where the initial CW hysteresis can be dynamically overcome by stronger counterclockwise (CCW) hysteresis towards memory-like dynamics. We systematically compare hysteresis in similar back-gated MoS$_2$/HfO$_2$ and MoS$_2$/Al$_2$O$_3$ FETs up to 275\textdegree C. At room temperature, both devices initially show sizable CW hysteresis. However, at 175\textdegree C MoS$_2$/HfO$_2$ FETs exhibit dominant CCW dynamics coupled with self-doping and negative differential resistance (NDR) effects. Our compact model suggests that this behavior is caused by the drift of mobile oxygen vacancies (\textit{V}${}_{\mathrm{O}}^{+}$ or \textit{V}${}_{\mathrm{O}}^{2+}$) within HfO$_2$ which also causes negative $V_{\mathrm{th}}$ shift under a constant positive bias stress. This alternative mechanism effectively overrides the initial CW hysteresis and enables intrinsic memory functionality that can be enhanced by using narrower gate bias sweep ranges. In contrast, the MoS$_2$/Al$_2$O$_3$ FETs display only minor CCW dynamics even at 275\textdegree C due to higher drift activation energies for the same vacancies, thereby maintaining superior stability. Our results reveal an insulators selection paradigm: Al$_2$O$_3$ layers are better suited to suppress detrimental negative $V_{\mathrm{th}}$ shifts in MoS$_2$ logic FETs at high temperatures, whereas their HfO$_2$ counterparts can serve as active memory layers that would exploit these abnormal instabilities.

cond-mat.mtrl-sci↗

arXiv2025

Anatomical Region-Guided Contrastive Decoding: A Plug-and-Play Strategy for Mitigating Hallucinations in Medical VLMs

Medical Vision-Language Models (MedVLMs) show immense promise in clinical applicability. However, their reliability is hindered by hallucinations, where models often fail to derive answers from visual evidence, instead relying on learned textual priors. Existing mitigation strategies for MedVLMs have distinct limitations: training-based methods rely on costly expert annotations, limiting scalability, while training-free interventions like contrastive decoding, though data-efficient, apply a global, untargeted correction whose effects in complex real-world clinical settings can be unreliable. To address these challenges, we introduce Anatomical Region-Guided Contrastive Decoding (ARCD), a plug-and-play strategy that mitigates hallucinations by providing targeted, region-specific guidance. Our module leverages an anatomical mask to direct a three-tiered contrastive decoding process. By dynamically re-weighting at the token, attention, and logits levels, it verifiably steers the model's focus onto specified regions, reinforcing anatomical understanding and suppressing factually incorrect outputs. Extensive experiments across diverse datasets, including chest X-ray, CT, brain MRI, and ocular ultrasound, demonstrate our method's effectiveness in improving regional understanding, reducing hallucinations, and enhancing overall diagnostic accuracy.

cs.CV↗

arXiv2025

DiffOP: Reinforcement Learning of Optimization-Based Control Policies via Implicit Policy Gradients

Real-world control systems require policies that are not only high-performing but also interpretable and robust. A promising direction toward this goal is model-based control, which learns system dynamics and cost functions from historical data and then uses these models to inform decision-making. Building on this paradigm, we introduce DiffOP, a novel framework for learning optimization-based control policies defined implicitly through optimization control problems. Without relying on value function approximation, DiffOP jointly learns the cost and dynamics models and directly optimizes the actual control costs using policy gradients. To enable this, we derive analytical policy gradients by applying implicit differentiation to the underlying optimization problem and integrating it with the standard policy gradient framework. Under standard regularity conditions, we establish that DiffOP converges to an $ε$-stationary point within $\mathcal{O}(ε^{-1})$ iterations. We demonstrate the effectiveness of DiffOP through experiments on nonlinear control tasks and power system voltage control with constraints. The code is available at https://github.com/alwaysbyx/DiffOP.

eess.SY↗

arXiv2025

Operator learning for energy-efficient building ventilation control with computational fluid dynamics simulation of a real-world classroom

Energy-efficient ventilation control plays a vital role in reducing building energy consumption while ensuring occupant health and comfort. While Computational Fluid Dynamics (CFD) simulations provide detailed and physically accurate representation of indoor airflow, their high computational cost limits their use in real-time building control. In this work, we present a neural operator learning framework that combines the physical accuracy of CFD with the computational efficiency of machine learning to enable building ventilation control with the high-fidelity fluid dynamics models. Our method jointly optimizes the airflow supply rates and vent angles to reduce energy use and adhere to air quality constraints. We train an ensemble of neural operator transformer models to learn the mapping from building control actions to airflow fields using high-resolution CFD data. This learned neural operator is then embedded in an optimization-based control framework for building ventilation control. Experimental results show that our approach achieves significant energy savings compared to maximum airflow rate control, rule-based control, as well as data-driven control methods using spatially averaged CO2 prediction and deep learning based reduced order model, while consistently maintaining safe indoor air quality. These results highlight the practicality and scalability of our method in maintaining energy efficiency and indoor air quality in real-world buildings.

eess.SY↗

arXiv2025

Stabilization of nonlinear systems with unknown delays via delay-adaptive neural operator approximate predictors

This work establishes the first rigorous stability guarantees for approximate predictors in delay-adaptive control of nonlinear systems, addressing a key challenge in practical implementations where exact predictors are unavailable. We analyze two scenarios: (i) when the actuated input is directly measurable, and (ii) when it is estimated online. For the measurable input case, we prove semi-global practical asymptotic stability with an explicit bound proportional to the approximation error $ε$. For the unmeasured input case, we demonstrate local practical asymptotic stability, with the region of attraction explicitly dependent on both the initial delay estimate and the predictor approximation error. To bridge theory and practice, we show that neural operators-a flexible class of neural network-based approximators-can achieve arbitrarily small approximation errors, thus satisfying the conditions of our stability theorems. Numerical experiments on two nonlinear benchmark systems-a biological protein activator/repressor model and a micro-organism growth Chemostat model-validate our theoretical results. In particular, our numerical simulations confirm stability under approximate predictors, highlight the strong generalization capabilities of neural operators, and demonstrate a substantial computational speedup of up to 15x compared to a baseline fixed-point method.

eess.SY↗