arXiv subjects

Yafei Wang

Publications and source records attributed to Yafei Wang.

At least 19 recordsLinked to original sources

arXiv2026

FirmCORe: A Benchmark for Structured Reasoning about Inter-Firm Collaboration Opportunities

Comprehensive structured data on inter-firm relationships is often scarce or inaccessible because many relationships are privately negotiated, selectively disclosed, and fragmented across proprietary databases. This scarcity hinders the discovery of collaboration opportunities, particularly for startups and small and medium-sized enterprises. Firm profiles are readily available, but collaboration potential cannot be inferred from business similarity alone, since similar firms may be competitors, whereas dissimilar firms may offer complementary products, technologies, channels, capabilities, or capital. We present FirmCORe (Inter-Firm Collaboration Opportunity Reasoning), a human-annotated benchmark for pairwise reasoning over weakly structured firm profiles, comprising 2,805 labeled firm pairs. Given two firm profiles, a model must determine whether the available evidence supports a collaboration opportunity and, for positive pairs, jointly predict its strength, primary collaboration type, and role direction. FirmCORe also provides parallel Chinese- and English-language evaluation sets containing identical instances and gold labels, enabling controlled analysis of input-language sensitivity. Experiments with representative locally deployed and hosted large language models (LLMs) show that the strongest model achieves a macro-F1 score of 74.51 for opportunity detection but only 61.57% exact match across all four output fields. Language effects vary across models, and high cross-language agreement can mask errors shared across languages. These results indicate that current LLMs are substantially more reliable at detecting broad collaboration opportunities than at identifying their specific types and role directions.

cs.AI↗

arXiv2026

From Prediction to Decision: World-Model-Guided Action Selection for Continuous Pile Excavation

Wheel-loader excavation is a sequential decision problem in which every scoop changes the terrain available to subsequent actions. A practical world model must predict action consequences accurately, rank candidates in real time, and operate inside the closed loop of a full-size machine. We present the World-Action Model (WAM), which proposes multiple scoops, rejects geometrically inadmissible candidates, jointly predicts signed terrain change and loaded volume, executes the candidate with the largest predicted load, and replans from the newly observed terrain. On 32 geometry-disjoint MinSlope test episodes, adding world-model ranking to matched diffusion proposals reduces the mean scoop count from 651.8 to 540.6 (17.1%), preserves 32/32 completion, and improves every paired episode. In a complete-system comparison, WAM completes 32/32 episodes versus 29/32 for an independently trained soft actor-critic policy. Comparisons of input representations, spatial support, and five architectures identify an accurate and efficient physics-structured predictor. We further evaluate the interface on event-disjoint full-size-loader data and deploy the complete perception-proposal-prediction-selection-execution loop for autonomous excavation. The ROS2/TensorRT implementation processes five candidates in 72.4 ms on a Jetson AGX Orin. The simulation results establish decision-level gains, while the physical experiments demonstrate real-world closed-loop feasibility.

cs.RO↗

arXiv2026

Robust Decentralized Multi-Satellite Massive MIMO Transmission via Knowledge Distillation

This paper investigates robust decentralized transmission for cooperative multi-satellite massive multiple-input multiple-output (MIMO) systems under imperfect statistical channel state information (sCSI). In the considered scenario, each satellite has complete access to its local information but receives partial information from other satellites due to limited inter-satellite links (ISLs), with only imperfect sCSI available. To address these challenges, we propose a knowledge distillation (KD) framework that transfers cooperative precoding knowledge from a centralized teacher neural network (NN) to lightweight decentralized student NNs. Specifically, a global-clean teacher, aggregating information from all satellites and accessing accurate sCSI during offline training, transfers its cooperative precoding knowledge to partial-noisy students, relying on complete local information, limited information exchanged by other satellites, and error-corrupted sCSI for local precoding. The teacher NN combines patch-wise self-attention with dual-axis attention to learn inter-user interference and inter-satellite coordination, whereas each student NN adopts a compact per-satellite architecture for efficient onboard inference. The teacher learns a high-quality weighted minimum mean square error precoding policy from global-clean inputs, which is then distilled into the students operating on partial-noisy inputs. To mitigate the resulting teacher-student performance gap, we develop a hybrid KD mechanism with explicit angle- and phase-error calibration. Simulation results demonstrate that the proposed framework significantly enhances the decentralized sum-rate performance and remains robust under diverse configurations.

eess.SP↗

arXiv2026

Toward Secure Communications for a UAV Swarm with Movable Antennas in SAGIN: CKM-Enabled Multi-Agent Reinforcement Learning Framework

Space-air-ground integrated networks (SAGINs) can provide ubiquitous and reliable connectivity for unmanned aerial vehicles (UAVs). However, air-to-ground links, which are typically dominated by line-of-sight (LoS) propagation, are vulnerable to passive eavesdropping due to the broadcast nature of wireless channels. To enhance physical-layer security, we investigate a SAGIN-enabled secure downlink communication system in which UAVs select service links among satellite, aerial, and terrestrial networks while adjusting the positions of the movable antenna (MA) array to fully exploit connectivity and spatial degrees of freedom for improved secrecy communication performance. Specifically, we maximize the secrecy energy efficiency (SEE) of a UAV swarm by jointly optimizing the MA positions, UAV trajectories, and link selections, subject to UAV mobility, MA movement, and link connectivity constraints. To reduce the real-time channel state information (CSI) acquisition overhead, we propose a channel knowledge map (CKM)-assisted multi-agent reinforcement learning framework. Specifically, the CKM is first constructed from sparse channel measurements via Kriging interpolation and is then leveraged together with satellite ephemeris information to enable efficient storage and retrieval of CSI. To reduce the action-space dimensionality and computational complexity, we model the MA array using rigid-body kinematics and adjust its position through global rigid-body translation, thereby constructing a low-dimensional hybrid action space for the joint optimization decisions. To align local decisions with system-wide performance under system constraints, we design an individual-team collaborative reward mechanism and introduce action masks to enforce constraints on UAV mobility, collision avoidance, MA regions, and connectivity capacity.

eess.SP↗

arXiv2026

Annually Modulated Pacific Subtropical Highs Pace the ENSO Cycle: A Hierarchical Framework from Earth Rotation to Atmosphere-Ocean Dynamics

The El Nino-Southern Oscillation drives Earth's dominant interannual climate signal, yet what initiates an event and what ends it remain incompletely understood. Here we show that anomalies of the eastern Pacific subtropical high (EPSH) belts, marched through the year by the annual cycle, precede and govern the equatorial zonal wind near the dateline, which sets subsequent sea surface temperature evolution. The Northern Hemisphere ridge triggers events in boreal spring, the Southern Hemisphere ridge sustains them in austral spring, and their synchrony decides ENSO diversity. Preceding the ridges lies a sub-annual oscillation in Earth's rotation rate (length of day, 0.5 to 1 yr band) that redistributes atmospheric mass poleward three times once its phase aligns with the annual cycle, in August, in December, and in the following May, each displacement carrying the response further east and poleward until the significant centers overlie the ridges themselves. Crucially, this band is deaf to ENSO: its cross-correlation with NINO3 never exceeds 0.05 at any lag, whereas the classical ENSO-driven rotation response resides in the 2 to 10 yr band at 0.40. The band therefore reads forward, and partial correlation identifies the EPSH as the conduit: the direct path from rotation to ocean vanishes once the ridges are held fixed. Complete by the July preceding the event year, the index fixes the phase of the coming event fourteen to seventeen months ahead, though not its amplitude. At maturity the event reorganizes the Hadley and Walker circulations into the Mirror-SIGMA pattern, disrupting the upstream chain and closing the cycle.

physics.ao-ph↗

arXiv2026

Toward Alias-Free Channel Extrapolation in Upper Mid-Band Systems: A Spatial-Frequency-Temporal Tensor Learning Approach

Upper mid-band massive multiple-input multiple-output (MIMO) offers a favorable capacity-coverage trade-off for next-generation wireless systems, but its large antenna arrays, wide bandwidths, and faster temporal variation substantially increase the pilot overhead required for accurate channel state information (CSI) acquisition. To reduce this overhead, this paper establishes a tensor-structured multi-domain channel extrapolation framework that exploits the limited-scattering nature of practical propagation environments to recover complete CSI across the spatial-frequency-temporal (SFT) domains from limited observations. Specifically, we develop a Tucker-based SFT-domain signal model to represent the complete CSI, where the factor matrices are parameterized by angle-delay-Doppler (ADD)-domain grids. Thanks to this representation, we reveal that limited SFT-domain observations imposed by uniform pilot patterns and antenna-port selection inherently induce ADD-domain aliasing, so that multiple physically distinct ADD-domain components become indistinguishable within structured ADD aliasing groups. To tackle this issue, we introduce a support-prior-assisted ADD-domain de-aliasing mechanism that leverages coarse-grained support information. Since exact closed-form characterization of this mechanism is difficult to derive, we propose a tensor-structure-aware axial-attention neural network (TANN), which integrates axis-wise attention with a lightweight multi-scale CNN-based gating module to incorporate support priors for ADD-domain de-aliasing. With tensor-structure modeling and mixed-configuration training over different pilot decimation factors, TANN yields a unified model that generalizes across pilot configurations without retraining. Numerical results demonstrate the effectiveness and strong generalization of the proposed framework over benchmark methods under diverse scenarios.

eess.SP↗

arXiv2026

Generative Actor-Critic with Soft Bridge Policies

Expressive generative policies such as diffusion and flow models are appealing for MaxEnt online reinforcement learning because of their ability to model multimodal and highly non-Gaussian action distributions. However, training effective soft generative policies faces two obstacles that often arise together. First, marginal action densities are often unavailable, so existing methods typically rely on entropy bounds, heuristic proxies or approximations. Second, iterative shared-parameter samplers raise inference cost and require backpropagation through time over repeated network evaluations, increasing memory cost and destabilizing policy optimization. These obstacles motivate us to seek a generative policy that exposes a tractable MaxEnt objective while requiring only a single sampled actor forward pass for action generation. To this end, we propose soft generative actor-critic (SoftGAC), whose actor defines a stochastic bridge from a fixed base latent to a terminal action latent in pre-tanh space. This structured bridge allows us to lift the MaxEnt objective as an analytically tractable path-wise relative-entropy objective against a high-entropy reference process. In practical finite-step implementation, this relative entropy reduces exactly to sampled transition control energy and thus provides principled soft regularization. Moreover, we keep the single-pass actor lightweight by using small step-specific bridge transitions, each evaluated only once per sampled action, while maintaining a parameter budget comparable to strong actor baselines. Extensive experiments on challenging continuous-control benchmarks show that SoftGAC attains higher or competitive returns than strong generative policy baselines, including diffusion and flow-matching policies, while staying in the low-latency regime of one-pass actors and showing considerable improvements in the compute-return tradeoff.

cs.LG↗

arXiv2026

Semantic Communication for Multi-Satellite Massive MIMO Transmission: A Mixture of Cooperative Modes Framework

This paper investigates semantic communications (SemComs) for multi-satellite cooperative massive multiple-input multiple-output (MIMO) transmission, where multiple massive-MIMO satellites jointly serve a common set of multi-antenna user terminals. For the first time, SemComs with image transmission task are integrated into satellite massive MIMO and multi-satellite cooperative transmission. For the two representative cooperative modes, namely coherent transmission (CT) and non-coherent transmission (NCT), we develop multi-satellite CT (MSCT) and multi-satellite NCT (MSNCT) SemCom frameworks, respectively. MSCT adopts a symmetric architecture, whereas MSNCT introduces transmitter-side stream allocation and a two-stage receiver design that combines per-stream semantic extraction with cross-stream semantic-interference exploitation. To instantiate MSCT, we further design a symmetric encoder and decoder network based on hybrid Swin-Transformer and lightweight bottleneck convolutional neural network (CNN) blocks, termed HSTC, where Swin Transformer provides scalable computation and the CNN branch improves performance and convergence. For MSNCT, a Transformer-based backbone is employed to support cross-stream interference exploitation through global attention. Building on these two frameworks, we propose a mixture of cooperative modes (MoCM) framework, in which a permutation-invariant network dynamically switches between MSCT and MSNCT using multi-satellite statistical channel state information, thereby balancing semantic performance and complexity. Simulation results under practical configurations demonstrate the performance gains of the proposed frameworks.

eess.SP↗

arXiv2026

Toward Multi-Satellite Cooperative Transmission: A Joint Framework for CSI Acquisition, Feedback, and Phase Synchronization

The stringent link budget, caused by long propagation distances and payload constraints, poses a fundamental bottleneck for single-satellite transmission. Although LEO mega-constellations make multi-satellite cooperative transmission (MSCT), such as distributed precoding (DP), increasingly feasible, its cooperative gains critically rely on stringent time-frequency-phase synchronization (TFP-Sync), which is difficult to maintain under rapid channel variation and feedback latency. To address this issue, this paper proposes a joint CSI acquisition, feedback, and phase-level synchronization (JCAFPS) framework for MSCT. Specifically, to enable reliable, overhead-efficient CSI acquisition, we design a beam-domain adjustable phase-shift tracking reference signal (TRS) transmission scheme, along with criteria for the TRS and CSI-feedback periods. Then, exploiting deterministic orbital motion and dominant LoS propagation, we establish a polynomial model for the temporal evolution of delay and Doppler shift, and derive an OFDM-based multi-satellite signal model under non-ideal synchronization. The analysis reveals that, unlike the single-satellite case, the composite multi-satellite channel exhibits nonlinear time-frequency-varying phase behavior, necessitating symbol- and subcarrier-wise phase precompensation for coherent transmission. Based on these results, we develop a practical closed-loop realization integrating single-TRS-based channel parameter estimation, multi-TRS-based channel prediction, predictive CSI feedback, and user-specific TFP precompensation. Numerical results demonstrate that the proposed framework achieves accurate CSI acquisition and precise TFP-Sync, enabling DP-based dual-satellite cooperative transmission to approach the theoretical 6 dB power gain over single-satellite transmission, while remaining robust under extended prediction durations and enlarged TRS periods.

eess.SP↗

arXiv2026

The People's Gaze: Co-Designing and Refining Gaze Gestures with General Users and Gaze Interaction Experts

As eye-tracking becomes increasingly common in modern mobile devices, the potential for hands-free, gaze-based interaction grows, but current gesture sets are largely expert-designed and often misaligned with how users naturally move their eyes. To address this gap, we introduce a two-phase methodology for developing intuitive gaze gestures. First, four co-design workshops with 20 non-expert participants generated 102 initial concepts. Next, four gaze interaction experts reviewed and refined these into a set of 32 gestures. We found that non-experts, after a brief introduction, intuitively anchor gestures in familiar metaphors and develop a compositional grammar; i.e., activation (dwell) + action (gaze gesture or blink), to ensure intentionality and mitigate the classic Midas Touch problem. Experts prioritized gestures that are ergonomically sound, aligned with natural saccades, and reliably distinguishable. The resulting user-grounded, expert-validated gesture set, along with actionable design principles, provides a foundation for developing intuitive, hands-free interfaces for gaze-enabled devices.

cs.HC↗

arXiv2026

Deep Learning-Based Multi-Satellite Massive MIMO Transmission: Centralized or Decentralized?

This paper investigates new efficient transmission architectures for multi-satellite massive multiple-input multiple-output (MIMO). We study the weighted sum-rate maximization problem in a multi-satellite system where multiple satellites transmit independent data streams to multi-antenna user terminals, thereby achieving higher throughput. We first adopt a multi-satellite weighted minimum mean square error (WMMSE) formulation under statistical channel state information (CSI), which yields closed-form updates for the precoding and receive vectors. To overcome the high complexity of optimization, we propose a learning-based WMMSE design that integrates tensor equivariance with closed-form recovery, enabling inference with near-optimal performance without iterative updates. Moreover, to reduce inter-satellite signaling overhead incurred by exchanging CSI and precoding vectors in centralized coordination, we develop a decentralized multi-satellite transmission scheme in which each satellite locally infers its precoders rather than receiving from the central satellite. The proposed decentralized scheme leverages periodically available satellite state information, such as orbital positions and satellite attitude, which is inherently accessible in satellite networks, and employs a dual-branch tensor-equivariant network to predict the precoders at each satellite locally. Numerical results demonstrate that the proposed multi-satellite transmission significantly outperforms single-satellite systems in sum rate; the decentralized scheme achieves sum-rate performance close to the centralized schemes while substantially reducing computational complexity and inter-satellite overhead; and the learning-based schemes exhibit strong robustness and scalability across different scenarios.

eess.SP↗

arXiv2026

Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving

Vision-Language Models (VLMs) have emerged as a promising paradigm in autonomous driving (AD), providing a unified framework for perception and decision-making. However, their real-world deployment is hindered by significant computational overhead when processing high-resolution, multi-view images. This complexity stems from the massive number of visual tokens, which increases inference latency and memory consumption due to the quadratic complexity of self-attention. To address these challenges, we propose Prune2Drive, a plug-and-play visual token pruning framework for multi-view VLMs in AD. Prune2Drive introduces two core innovations: (i) a diversity-aware token selection mechanism that prioritizes semantic and spatial coverage across views, and (ii) a view-adaptive pruning controller that automatically learns optimal pruning ratios based on camera importance to downstream tasks. Unlike prior methods, Prune2Drive requires no model retraining or access to attention maps, ensuring compatibility with modern efficient attention implementations. Extensive experiments on the DriveLM and DriveLMM-o1 benchmarks demonstrate that Prune2Drive achieves significant speedups and memory savings with minimal performance impact. When retaining only 10% of visual tokens, our method achieves a 6.40x speedup in the prefilling phase and consumes only 13.4% of the original FLOPs, with a mere 3% average performance drop on the DriveLM benchmark. Code is available at: https://github.com/MinhaoXiong/Prune2Drive.git

cs.CV↗

arXiv2025

Multi-Satellite Multi-Stream Beamspace Massive MIMO Transmission

This paper studies multi-satellite multi-stream (MSMS) beamspace transmission, where multiple satellites cooperate to form a distributed multiple-input multiple-output (MIMO) system and jointly deliver multiple data streams to multi-antenna user terminals (UTs), and beamspace transmission combines earth-moving beamforming with beam-domain precoding. For the first time, we formulate the signal model for MSMS beamspace MIMO transmission. Under synchronization errors, multi-antenna UTs enable the distributed MIMO channel to exhibit higher rank, supporting multiple data streams. Beamspace MIMO retains conventional codebook based beamforming while providing the performance gains of precoding. Based on the signal model, we propose statistical channel state information (sCSI)-based optimization of satellite clustering, beam selection, and transmit precoding, using a sum-rate upper-bound approximation. With given satellite clustering and beam selection, we cast precoder design as an equivalent covariance decomposition-based weighted minimum mean square error (CDWMMSE) problem. To obtain tractable algorithms, we develop a closed-form covariance decomposition required by CDWMMSE and derive an iterative MSMS beam-domain precoder under sCSI. Following this, we further propose several heuristic closed-form precoders to avoid iterative cost. For satellite clustering, we enhance a competition-based algorithm by introducing a mechanism to regulate the number of satellites serving certain UT. Furthermore, we design a two-stage low-complexity beam selection algorithm focused on enhancing the effective channel power. Simulations under practical configurations validate the proposed methods across the number of data streams, receive antennas, serving satellites, and active beams, and show that beamspace transmission approaches conventional MIMO performance at lower complexity.

eess.SP↗

arXiv2025

Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning

The remarkable empirical performance of distributional reinforcement learning (RL) has garnered increasing attention to understanding its theoretical advantages over classical RL. By decomposing the categorical distributional loss commonly employed in distributional RL, we find that the potential superiority of distributional RL can be attributed to a derived distribution-matching entropy regularization. This less-studied entropy regularization aims to capture additional knowledge of return distribution beyond only its expectation, contributing to an augmented reward signal in policy optimization. In contrast to the vanilla entropy regularization in MaxEnt RL, which explicitly encourages exploration by promoting diverse actions, the novel entropy regularization derived from categorical distributional loss implicitly updates policies to align the learned policy with (estimated) environmental uncertainty. Finally, extensive experiments verify the significance of this uncertainty-aware regularization from distributional RL on the empirical benefits over classical RL. Our study offers an innovative exploration perspective to explain the intrinsic benefits of distributional learning in RL.

cs.LG↗

arXiv2025

QoS-Aware Hierarchical Reinforcement Learning for Joint Link Selection and Trajectory Optimization in SAGIN-Supported UAV Mobility Management

Due to the significant variations in unmanned aerial vehicle (UAV) altitude and horizontal mobility, it becomes difficult for any single network to ensure continuous and reliable threedimensional coverage. Towards that end, the space-air-ground integrated network (SAGIN) has emerged as an essential architecture for enabling ubiquitous UAV connectivity. To address the pronounced disparities in coverage and signal characteristics across heterogeneous networks, this paper formulates UAV mobility management in SAGIN as a constrained multi-objective joint optimization problem. The formulation couples discrete link selection with continuous trajectory optimization. Building on this, we propose a two-level multi-agent hierarchical deep reinforcement learning (HDRL) framework that decomposes the problem into two alternately solvable subproblems. To map complex link selection decisions into a compact discrete action space, we conceive a double deep Q-network (DDQN) algorithm in the top-level, which achieves stable and high-quality policy learning through double Q-value estimation. To handle the continuous trajectory action space while satisfying quality of service (QoS) constraints, we integrate the maximum-entropy mechanism of the soft actor-critic (SAC) and employ a Lagrangian-based constrained SAC (CSAC) algorithm in the lower-level that dynamically adjusts the Lagrange multipliers to balance constraint satisfaction and policy optimization. Moreover, the proposed algorithm can be extended to multi-UAV scenarios under the centralized training and decentralized execution (CTDE) paradigm, which enables more generalizable policies. Simulation results demonstrate that the proposed scheme substantially outperforms existing benchmarks in throughput, link switching frequency and QoS satisfaction.

eess.SP↗

arXiv2025

Deep Learning-Based Joint Uplink-Downlink CSI Acquisition for Next-Generation Upper Mid-Band Systems

In next-generation wireless communication systems, the newly designated upper mid-band has attracted considerable attention, also called frequency range 3 (FR3), highlighting the need for downlink (DL) transmission design, which fundamentally relies on accurate CSI. However, CSI acquisition in FR3 systems faces significant challenges: the increased number of antennas and wider transmission bandwidth introduces prohibitive training overhead with traditional estimation approaches, as each probing captures only incomplete spatial-frequency observation, while higher carrier frequencies lead to faster temporal channel variation. To address these challenges, we propose a novel CSI acquisition framework that integrates CSI feedback, uplink (UL) and DL channel estimation, as well as channel prediction in the FR3 TDD massive MIMO systems. Specifically, we first develop the Joint UL and DL Channel Estimation Network (JUDCEN) to fuse incomplete observations based on the SRSs and CSI-RSs. By exploiting the complementary characteristics of preliminary UL and DL estimation features, obtained through initial UL estimation and quantized-feedback-assisted DL estimation, it enables full CSI reconstruction in the spatial domain. To mitigate the performance degradation in the feedback process, we propose the Transformer-MLP CSI Feedback Network (TMCFN), employing an MLP-based module to jointly exploit angle- and delay-domain features. Building upon the reconstructed full CSI, we further develop the Mamba-based Channel Prediction Network (MCPN), which exploits selective state-space model (SSM) mechanism to capture long-range temporal dynamics in the angle-delay domain for future CSI prediction. Simulation results demonstrate that the proposed framework consistently outperforms benchmarks in both CSI acquisition accuracy and transmission spectral efficiency with lower computational complexity.

eess.SP↗

arXiv2025

Unlocking Symbol-Level Precoding Efficiency Through Tensor Equivariant Neural Network

Although symbol-level precoding (SLP) based on constructive interference (CI) exploitation offers performance gains, its high complexity remains a bottleneck. This paper addresses this challenge with an end-to-end deep learning (DL) framework with low inference complexity that leverages the structure of the optimal SLP solution in the closed-form and its inherent tensor equivariance (TE), where TE denotes that a permutation of the input induces the corresponding permutation of the output. Building upon the computationally efficient model-based formulations, as well as their known closed-form solutions, we analyze their relationship with linear precoding (LP) and investigate the corresponding optimality condition. We then construct a mapping from the problem formulation to the solution and prove its TE, based on which the designed networks reveal a specific parameter-sharing pattern that delivers low computational complexity and strong generalization. Leveraging these, we propose the backbone of the framework with an attention-based TE module, achieving linear computational complexity. Furthermore, we demonstrate that such a framework is also applicable to imperfect CSI scenarios, where we design a TE-based network to map the CSI, statistics, and symbols to auxiliary variables. Simulation results show that the proposed framework captures substantial performance gains of optimal SLP, while achieving an approximately 80-times speedup over conventional methods and maintaining strong generalization across user numbers and symbol block lengths.

eess.SP↗

arXiv2025

Toward Optimal Statistical Inference in Noisy Linear Quadratic Reinforcement Learning over a Finite Horizon

Recent developments in Reinforcement learning have significantly enhanced sequential decision-making in uncertain environments. Despite their strong performance guarantees, most existing work has focused primarily on improving the operational accuracy of learned control policies and the convergence rates of learning algorithms, with comparatively little attention to uncertainty quantification and statistical inference. Yet, these aspects are essential for assessing the reliability and variability of control policies, especially in high-stakes applications. In this paper, we study statistical inference for the policy gradient (PG) method for noisy Linear Quadratic Reinforcement learning (LQ RL) over a finite time horizon, where linear dynamics with both known and unknown drift parameters are controlled subject to a quadratic cost. We establish the theoretical foundations for statistical inference in LQ RL, deriving exact asymptotics for both the PG estimators and the corresponding objective loss. Furthermore, we introduce a principled inference framework that leverages online bootstrapping to construct confidence intervals for both the learned optimal policy and the corresponding objective losses. The method updates the PG estimates along with a set of randomly perturbed PG estimates as new observations arrive. We prove that the proposed bootstrapping procedure is distributionally consistent and that the resulting confidence intervals achieve both asymptotic and non-asymptotic validity. Notably, our results imply that the quantiles of the exact distribution can be approximated at a rate of $n^{-1/4}$, where $n$ is the number of samples used during the procedure. The proposed procedure is easy to implement and applicable to both offline and fully online settings. Numerical experiments illustrate the effectiveness of our approach across a range of noisy linear dynamical systems.

math.ST↗