arXiv subjects

Nian Si

Publications and source records attributed to Nian Si.

At least 19 recordsLinked to original sources

arXiv2026

Asymptotic Optimality of $cμ$-Rule for LLM Inference Scheduling under Memory Constraint

Large language model (LLM) inference, the process by which a pre-trained model receives an input prompt and generates output tokens sequentially, has become increasingly important with the widespread deployment of LLM-based applications. As inference workloads continue to scale, GPU memory has emerged as a critical bottleneck: the long input prompts and generation sequences common in agentic and reasoning workloads lead to substantial key-value (KV) cache usage. In this paper, we study the resulting online scheduling problem from a queueing perspective. Requests arrive stochastically with heterogeneous prompt and output lengths, undergo prefill and autoregressive decode, and compete for a finite KV-cache capacity. Unlike classical queueing models, the duration of a processing step depends on both prefill computation and KV-cache I/O, so service capacity is jointly determined by computation, memory access, and memory occupancy. We develop a fluid model that reduces the system to a multiclass single-server queue with effective service rates capturing computation, memory access, and memory capacity. This fluid analysis yields a $cμ$ priority rule, where the effective service rate $μ$ is jointly determined by computational speed, memory bandwidth, and memory capacity. We prove that this rule minimizes cumulative holding cost in the fluid model and, for a finite workload, also minimizes completion time and maximizes full-run output-token throughput. We then construct a feasible scheduling policy for the original finite system and prove that its scaled dynamics converge to the optimal fluid policy. Finally, experiments on a real LLM serving engine using an NVIDIA H800 show that practical implementations of the proposed priority rule substantially reduce average latency while maintaining high throughput relative to existing scheduling policies.

math.OC↗

arXiv2026

Learning Sequential Decisions from Multiple Sources via Group-Robust Markov Decision Processes

Learning a common policy from multiple source environments requires accounting for cross-source heterogeneity, and the offline setting introduces an additional challenge of limited data coverage. We formulate this problem as a distributionally robust Markov decision process under a shared linear MDP representation with source-specific transition and reward components. The uncertainty set is constructed through feature-wise mixtures of these components, yielding a $d$-rectangular structure that captures cross-source heterogeneity and admits a robust Bellman recursion. Building on this formulation, we develop Group-Robust Pessimistic Value Iteration (GR-PEVI), an offline policy-learning algorithm that combines feature-wise worst-case aggregation with a data-dependent pessimism penalty. Under a robust partial coverage condition, we establish finite-sample suboptimality guarantees for GR-PEVI and extend the analysis to clustered sources, characterizing the trade-off between the statistical gain from pooling similar sources and the approximation error induced by within-cluster heterogeneity. Simulation studies demonstrate the effectiveness of GR-PEVI, with empirical behavior consistent with the theoretical analysis. We apply GR-PEVI to a multi-source electronic health record study of multiple sclerosis treatment decisions to illustrate its use in a clinical setting.

stat.ME↗

arXiv2026

Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gain

Learning and optimal control under robust Markov decision processes (MDPs) have received increasing attention, yet most existing theory, algorithms, and applications focus on finite-horizon or discounted models. Long-run average-reward formulations, while natural in many operations research and management contexts, remain underexplored. This is primarily because the dynamic programming foundations are technically challenging and only partially understood, with several fundamental questions remaining open. This paper steps toward a general framework for average-reward robust MDPs by analyzing the constant-gain setting. We study the average-reward robust control problem with possible information asymmetries between the controller and an S-rectangular adversary. Our analysis centers on the constant-gain robust Bellman equation, examining both the existence of solutions and their relationship to the optimal average reward. Specifically, we identify when solutions to the robust Bellman equation characterize the optimal average reward and stationary policies, and we provide one-sided weak communication conditions ensuring solutions' existence. These findings expand the dynamic programming theory for average-reward robust MDPs and lay a foundation for robust dynamic decision making under long-run average criteria in operational environments.

math.OC↗

arXiv2026

Distributionally Robust Average-Reward Reinforcement Learning: Finite-Sample Guarantees under Weak Communication

We study distributionally robust reinforcement learning (DR-RL) in the average-reward setting under weak communication. Our main result provides finite-sample guarantees for estimating the robust optimal average reward and learning a near-optimal policy, covering both SA-rectangular and S-rectangular structures with divergence-based and distance-based uncertainty sets. Specifically, for Kullback--Leibler and $f_k$-divergence balls, we establish explicit radius conditions under which the robust average-reward Bellman equation admits a constant-gain solution, while for total variation and Wasserstein balls, any positive radius suffices without requiring the nominal MDP to be weakly communicating. Our algorithm is prior-knowledge-free and achieves sample complexities of $\widetilde O(|\mathcal{S}||\mathcal{A}|p_{\wedge}^{-1}\operatorname{Span}^{2}(u_δ^{\ast})ε^{-2})$ for estimating the robust optimal average reward and $\widetilde O(|\mathcal{S}||\mathcal{A}|p_{\wedge}^{-2}\operatorname{Span}^{2}(u_δ^{\ast})ε^{-2})$ for learning an $ε$-optimal policy. Here, $p_{\wedge}$ is the smallest positive nominal transition probability and $u_δ^{\ast}$ is a robust optimal bias function. We further provide an almost-tight explicit upper bound on $\operatorname{Span}(u_δ^{\ast})$. Finally, we validate the predicted $n^{-1/2}$ convergence rate through numerical experiments.

cs.LG↗

arXiv2026

Dynamic Programming-Compatible Uncertainty Sets in Robust Markov Decision Processes

In this paper, we investigate the compatibility of robust Markov Decision Processes (RMDPs) with dynamic programming under various assumptions on the uncertainty set, i.e., we investigate when one can solve an RMDP by solving a fixed point equation. We show that in all generality, s-rectangular and sa-rectangular uncertainty sets are the only models of uncertainty that are compatible with dynamic programming. Our analysis shows that existing non-rectangular models, including r-rectangularity, are only weakly compatible with dynamic programming, as they require the assumption that rewards do not depend on the next state. In this case, our results imply that one can always construct a rectangular uncertainty set that is equivalent, for both policy evaluation and optimization, to the dynamic programming-compatible non-rectangular model. This highlights a key limitation: dynamic-programming-compatible non-rectangular uncertainty sets, although practically relevant for uncertainty quantification, do not provide a genuinely distinct assessment of policy performance. Interestingly, our proof techniques rely on identifying a novel simultaneous solvability property, which we show is central to several important properties of RMDPs, including the existence of stationary optimal policies and dynamic programming-based formulations. The simultaneous solvability property enables a unified approach to studying all existing models of uncertainty, rectangular and non-rectangular alike.

math.OC↗

arXiv2026

Diffusion-Based Policies for Dynamic Control of Stochastic Processing Networks

We consider a processing network model with $m$ job classes or buffers, exogenous input flows into some classes, $n$ processing activities, and $p$ servers. Each activity is either a specified server processing jobs of a specified class, or a fictional input server delivering jobs of a specified class; jobs change class in Markovian fashion after completing service. A standard multiclass queueing network, with its one-to-one correspondence between job classes and activities, is a special case, but our general model allows two or more ways to process a given class, and some or all input flows may be turned away at the system manager's discretion. Costs are linear: a holding cost per time unit for each class $i$ job in the system, and a rejection penalty for each class $i$ arrival denied access $(i=1,\ldots,m)$. The system manager makes input control, job routing, and order-of-service decisions to minimize expected discounted costs over an infinite horizon. We formulate an approximating Brownian control problem (BCP) whose state space is the $m$-dimensional nonnegative orthant; control is a drift vector chosen from a bounded polyhedral set, based on dynamic state observations. Using recently developed computational methods, the BCP can be solved numerically in dimensions up to at least $m=50$, and we explain how the numerical solution is translated into an implementable control policy for the queueing system of original interest. Previous work on heavy traffic diffusion approximations suggests that this policy is nearly optimal in the heavy traffic parameter regime, and numerical examples support that conjecture. We also discuss its advantage over an alternative approach, featured in our previous work, where the BCP is replaced by a lower-dimensional "equivalent workload formulation" that is computationally efficient but difficult to interpret in the network of original interest.

math.OC↗

arXiv2026

Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework

Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions. However, standard DRO formulations often treat all feature perturbations equally. This can be unnecessarily conservative when external knowledge suggests that the predictive signal is embedded in a low-dimensional representation of covariates. We propose REpresentation-Aware Distributionally robust estimation (READ), a Wasserstein DRO framework that uses external representations to guide the geometry of robustness. Rather than uniformly perturbing all covariate directions, READ increases the transport cost of perturbations that change representation coordinates, thereby reshaping the dual regularization toward the representation subspace. Meanwhile, it preserves protection against variations orthogonal to the representation. We study READ in two regimes. First, for inference on the current target, we characterize our estimator asymptotically and develop a Wasserstein profile inference approach to construct representation-aligned confidence regions while enabling automatic hyperparameter tuning. Second, for deployment to future populations that differ from the current target but are generated from the same representation-invariant random-coefficient model, we show that the resulting regions achieve higher coverage of future model parameters than standard methods. Simulations and a single-cell multi-omics application demonstrate the advantages of READ in multi-source and multitask transfer learning settings.

stat.ME↗

arXiv2026

Experimental Designs for Multi-Item Multi-Period Inventory Control

Randomized experiments, or A/B testing, are the gold standard for evaluating interventions, yet they remain underutilized in inventory management. This study addresses this gap by analyzing A/B testing strategies in multi-item, multi-period inventory systems with lost sales and capacity constraints. We examine two canonical experimental designs--switchback experiments and item-level randomization--and show that both suffer from systematic bias due to interference: temporal carryover in switchbacks and cannibalization across items under capacity constraints. Under mild conditions, we characterize the direction of this bias in different scenarios. Motivated by two-sided randomization, we propose a pairwise design over items and time and analyze its bias properties. Controlled stochastic simulations verify the theoretical predictions, and trace-driven experiments on real-world fresh-retail data show that the same mechanisms persist in realistic environments with stockout substitution.

stat.ME↗

arXiv2026

Causal Estimation of Share-Induced Engagement with Flywheel Effects

Sustainable user growth in online platforms depends not only on acquiring new users but also on reactivating and engaging existing ones through social sharing features. A well-designed sharing feature can trigger a self-reinforcing ``flywheel effect'': reactivated users become potential sharers whose engagement propagates through the network over multiple rounds, amplifying total engagement. Measuring the causal impact of such sharing features is challenging, as their effects unfold through complex social networks and temporal cascades, violating the no-interference assumption underlying classical A/B testing. We develop a framework for experiments on sharing features that accounts for interference caused by the flywheel effect and targets a global treatment effect on share-induced engagement. Our estimator is motivated by a flow-balance identity and interprets share-induced engagement as a geometric amplification process, yielding a closed-form propagation adjustment that accounts for multi-round diffusion using commonly available attribution logs. Under mild conditions, we establish consistency of the proposed estimator and develop a valid A/A testing procedure for pipeline validation. Simulation studies show that our method substantially reduces bias relative to the difference-in-means estimator and first-order adjustments, while the proposed A/A test maintains nominal Type I error. We also extend the framework to a user-level reactivation metric via a Poisson approximation. Finally, we demonstrate the approach on a real-world large-scale online platform and discuss empirical implications for evaluating sharing feature designs.

stat.ME↗

arXiv2026

Experimentation for Different Scheduling Policies on Queues: Mixed Differences-in-Q Estimators Based on Little's Law

In data centers, tasks are dispatched to various servers to evenly distribute the workload. When a data center considers implementing a new scheduling algorithm, it typically conducts an A/B test prior to deployment to assess the real-world impact of this new method. However, a straightforward A/B test might be interfered with so-called ``Markovian'' interference. We utilized the Differences-in-Q estimator, as developed by Farias et al. (2022), and introduced mixed Differences-in-Q estimators grounded in Little's Law. We show that our A/B testing methods significantly reduce bias and variance when testing various scheduling policies. Extensive simulations were conducted under scenarios like non-stationary arrival rates, heterogeneous service rates, and communication delays. These simulations highlight the robustness and efficacy of our A/B testing approach.

stat.ME↗

arXiv2026

A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints

The rapid adoption of large language models (LLMs) has created significant challenges for efficient inference at scale. Unlike traditional workloads, LLM inference is constrained by both computation and the memory overhead of key-value (KV) caching, which accelerates decoding but quickly exhausts GPU memory. In this paper, we introduce the first queueing-theoretic framework that explicitly incorporates both computation and GPU memory constraints into the analysis of LLM inference. Based on this framework, we derive rigorous stability and instability conditions that determine whether an LLM inference service can sustain incoming demand without unbounded queue growth. This result offers a powerful tool for system deployment, potentially addressing the core challenge of GPU provisioning. By combining an estimated request arrival rate with our derived stable service rate, operators can calculate the necessary cluster size to avoid both costly over-purchasing and performance-violating under-provisioning. We further validate our theoretical predictions through extensive experiments in real GPU production environments. Our results show that the predicted stability conditions are highly accurate, with deviations typically within 10%.

cs.LG↗

arXiv2026

Near-Optimal Sample Complexities of Divergence-based S-rectangular Distributionally Robust Reinforcement Learning

Distributionally robust reinforcement learning (DR-RL) has recently gained significant attention as a principled approach that addresses discrepancies between training and testing environments. To balance robustness, conservatism, and computational traceability, the literature has introduced DR-RL models with SA-rectangular and S-rectangular adversaries. While most existing statistical analyses focus on SA-rectangular models, owing to their algorithmic simplicity and the optimality of deterministic policies, S-rectangular models more accurately capture distributional discrepancies in many real-world applications and often yield more effective robust randomized policies. In this paper, we study the empirical value iteration algorithm for divergence-based S-rectangular DR-RL and establish near-optimal sample complexity bounds of $\widetilde{O}(|\mathcal{S}||\mathcal{A}|(1-γ)^{-4}\varepsilon^{-2})$, where $\varepsilon$ is the target accuracy, $|\mathcal{S}|$ and $|\mathcal{A}|$ denote the cardinalities of the state and action spaces, and $γ$ is the discount factor. To the best of our knowledge, these are the first sample complexity results for divergence-based S-rectangular models that achieve optimal dependence on $|\mathcal{S}|$, $|\mathcal{A}|$, and $\varepsilon$ simultaneously. We further validate this theoretical dependence through numerical experiments on a robust inventory control problem and a theoretical worst-case example, demonstrating the fast learning performance of our proposed algorithm.

cs.LG↗

arXiv2026

Non-Rectangular Average-Reward Robust MDPs: Optimal Policies and Their Transient Values

We study non-rectangular robust Markov decision processes under the average-reward criterion, where the ambiguity set couples transition probabilities across states and the adversary commits to a stationary kernel for the entire horizon. We show that any history-dependent policy achieving sublinear expected regret uniformly over the ambiguity set is robust-optimal, and that the robust value admits a minimax representation as the infimum over the ambiguity set of the classical optimal gains, without requiring any form of rectangularity or robust dynamic programming principle. Under the weak communication assumption, we establish the existence of such policies by converting high-probability regret bounds from the average-reward reinforcement learning literature into the expected-regret criterion. We then introduce a transient-value framework to evaluate finite-time performance of robust optimal policies, proving that average-reward optimality alone can mask arbitrarily poor transients and deriving regret-based lower bounds on transient values. Finally, we construct an epoch-based policy that combines an optimal stationary policy for the worst-case model with an anytime-valid sequential test and an online learning fallback, achieving a constant-order transient value.

math.OC↗

arXiv2026

Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learning

Motivated by practical applications where stable long-term performance is critical-such as robotics, operations research, and healthcare-we study the problem of distributionally robust (DR) average-reward reinforcement learning. We propose two algorithms that achieve near-optimal sample complexity. The first reduces the problem to a DR discounted Markov decision process (MDP), while the second, Anchored DR Average-Reward MDP, introduces an anchoring state to stabilize the controlled transition kernels within the uncertainty set. Assuming the nominal MDP is uniformly ergodic, we prove that both algorithms attain a sample complexity of $\widetilde{O}\left(|\mathbf{S}||\mathbf{A}| t_{\mathrm{mix}}^2\varepsilon^{-2}\right)$ for estimating the optimal policy as well as the robust average reward under KL and $f_k$-divergence-based uncertainty sets, provided the uncertainty radius is sufficiently small. Here, $\varepsilon$ is the target accuracy, $|\mathbf{S}|$ and $|\mathbf{A}|$ denote the sizes of the state and action spaces, and $t_{\mathrm{mix}}$ is the mixing time of the nominal MDP. This represents the first finite-sample convergence guarantee for DR average-reward reinforcement learning. We further validate the convergence rates of our algorithms through numerical experiments.

cs.LG↗

arXiv2026

Achieving $\varepsilon^{-2}$ Dependence for Average-Reward Q-Learning with a New Contraction Principle

We present the convergence rates of synchronous and asynchronous Q-learning for average-reward Markov decision processes, where the absence of contraction poses a fundamental challenge. Existing non-asymptotic results overcome this challenge by either imposing strong assumptions to enforce seminorm contraction or relying on discounted or episodic Markov decision processes as successive approximations, which either require unknown parameters or result in suboptimal sample complexity. In this work, under a reachability assumption, we establish optimal $\widetilde{O}(\varepsilon^{-2})$ sample complexity guarantees (up to logarithmic factors) for a simple variant of synchronous and asynchronous Q-learning that samples from the lazified dynamics, where the system remains in the current state with some fixed probability. At the core of our analysis is the construction of an instance-dependent seminorm and showing that, after a lazy transformation of the Markov decision process, the Bellman operator becomes one-step contractive under this seminorm.

cs.LG↗

arXiv2026

FutureX-Pro: Extending Future Prediction to High-Value Vertical Domains

Building upon FutureX, which established a live benchmark for general-purpose future prediction, this report introduces FutureX-Pro, including FutureX-Finance, FutureX-Retail, FutureX-PublicHealth, FutureX-NaturalDisaster, and FutureX-Search. These together form a specialized framework extending agentic future prediction to high-value vertical domains. While generalist agents demonstrate proficiency in open-domain search, their reliability in capital-intensive and safety-critical sectors remains under-explored. FutureX-Pro targets four economically and socially pivotal verticals: Finance, Retail, Public Health, and Natural Disaster. We benchmark agentic Large Language Models (LLMs) on entry-level yet foundational prediction tasks -- ranging from forecasting market indicators and supply chain demands to tracking epidemic trends and natural disasters. By adapting the contamination-free, live-evaluation pipeline of FutureX, we assess whether current State-of-the-Art (SOTA) agentic LLMs possess the domain grounding necessary for industrial deployment. Our findings reveal the performance gap between generalist reasoning and the precision required for high-value vertical applications.

cs.AI↗

arXiv2025

Learning Optimal Distributionally Robust Stochastic Control in Continuous State Spaces

We study data-driven learning of robust stochastic control for infinite-horizon systems with potentially continuous state and action spaces. In many managerial settings--supply chains, finance, manufacturing, services, and dynamic games--the state-transition mechanism is determined by system design, while available data capture the distributional properties of the stochastic inputs from the environment. For modeling and computational tractability, a decision maker often adopts a Markov control model with i.i.d. environment inputs, which can render learned policies fragile to internal dependence or external perturbations. We introduce a distributionally robust stochastic control paradigm that promotes policy reliability by introducing adaptive adversarial perturbations to the environment input, while preserving the modeling, statistical, and computational tractability of the Markovian formulation. From a modeling perspective, we examine two adversary models--current-action-aware and current-action-unaware--leading to distinct dynamic behaviors and robust optimal policies. From a statistical learning perspective, we characterize optimal finite-sample minimax rates for uniform learning of the robust value function across a continuum of states under ambiguity sets defined by the $f_k$-divergence and Wasserstein distance. To efficiently compute the optimal robust policies, we further propose algorithms inspired by deep reinforcement learning methodologies. Finally, we demonstrate the applicability of the framework to real managerial problems.

stat.ML↗

arXiv2025

Selecting the Best Optimizing System

We formulate selecting the best optimizing system (SBOS) problems and provide solutions for those problems. In an SBOS problem, a finite number of systems are contenders. Inside each system, a continuous decision variable affects the system's expected performance. An SBOS problem compares different systems based on their expected performances under their own optimally chosen decision to select the best, without advance knowledge of expected performances of the systems nor the optimizing decision inside each system. We design easy-to-implement algorithms that adaptively chooses a system and a choice of decision to evaluate the noisy system performance, sequentially eliminates inferior systems, and eventually recommends a system as the best after spending a user-specified budget. The proposed algorithms integrate the stochastic gradient descent method and the sequential elimination method to simultaneously exploit the structure inside each system and make comparisons across systems. For the proposed algorithms, we prove exponential rates of convergence to zero for the probability of false selection, as the budget grows to infinity. We conduct three numerical examples that represent three practical cases of SBOS problems. Our proposed algorithms demonstrate consistent and stronger performances in terms of the probability of false selection over benchmark algorithms under a range of problem settings and sampling budgets.

stat.ME↗