arXiv subjects

Negar Kiyavash

Publications and source records attributed to Negar Kiyavash.

At least 19 recordsLinked to original sources

arXiv2026

Superquantile-Gibbs Relaxation for Minima-selection in Bilevel Optimization

Bilevel optimization (BLO) becomes more challenging when the lower-level objective admits multiple minimizers. Compared with the commonly studied unique-minimizer setting, this introduces two difficulties: (1) evaluating the hyper-objective $F_{\max}$ requires minima selection over the lower-level solution set; and (2) $F_{\max}$ may be discontinuous without additional structure. We address both issues under a parameter-uniform local Polyak-Lojasiewicz (PL) condition on the lower-level objective, denoted by $\mathrm{PL}^{\circ}$. Unlike the global PL condition often assumed in BLO, the PL inequality in $\mathrm{PL}^{\circ}$ is imposed only near the local minima. This formulation accommodates bounded, non-singleton minimizer sets and is motivated by hyperparameter tuning in over-parameterized learning. We show that $F_{\max}$ is Lipschitz continuous and that the lower-level minimizer sets are connected, compact, embedded submanifolds with a common intrinsic dimension $k$. This intrinsic dimension determines the explicit accuracy exponents in our complexity bound. We propose a randomized method whose output satisfies an $(\mathcal O(ε),ρ)$-Goldstein stationarity condition for $F_{\max}$ in expectation. In the small-accuracy regime, suppressing logarithmic and fixed-problem factors, it uses at most $\widetilde{\mathcal O}(m^{15+12k}ε^{-2}(ερ)^{-12k-14})$ queries to the gradient oracle of the lower-level objective, where $m$ is the upper-level dimension. Its key ingredient is a Superquantile-Gibbs relaxation that provides an approximate function-evaluation oracle for $F_{\max}$ and turns minima selection into a sampling problem addressed by standard Langevin dynamics. To our knowledge, this is the first rigorous treatment of minima selection in BLO that explicitly relates overall complexity to the lower-level intrinsic dimension.

math.OC↗

arXiv2026

Sharp First-Order Lower Bounds under $α$-Polyak-Lojasiewicz Conditions

We study first-order oracle complexity under the $α$-Polyak-Lojasiewicz condition $f(x)-f_\star\leq τ\|\nabla f(x)\|^α$ for $α\in[1,2]$. For $α<2$, we first show that global $L$-smoothness together with a global $α$-Polyak-Lojasiewicz inequality forces the objective to be constant. This motivates a nontrivial model in which smoothness remains global but the inequality is required only on the initial sublevel set. On this class, we establish sharp minimax lower bounds for every $α\in[1,2)$. Deterministic first-order methods require $Ω(Lτ^{2/α}\varepsilon^{-(2-α)/α})$ oracle calls, matching gradient descent. With unbiased stochastic gradients of conditional variance at most $σ^2$, randomized first-order methods require $Ω(Lτ^{2/α}\varepsilon^{-(2-α)/α}+Lσ^2τ^{4/α}\varepsilon^{-(4-α)/α})$ calls, matching the corresponding SGD dependence when the inequality holds along the stochastic trajectory. At the classical endpoint $α=2$, a separate construction yields the variance-dependent lower bound $Ω(Lτ^2σ^2/\varepsilon)$ even for globally smooth objectives satisfying the Polyak-Lojasiewicz inequality globally. In contrast, the sharp variance-dependent complexity for smooth $μ$-strongly convex objectives is $Θ(σ^2/(μ\varepsilon))$; with $τ=(2μ)^{-1}$, the worst-case global Polyak-Lojasiewicz class exhibits an additional factor $Lτ=L/(2μ)$.

math.OC↗

arXiv2026

Active Context Selection Improves Simple Regret in Contextual Bandits

We study the contextual multi-armed bandit problem with a finite context space (a.k.a. subpopulations), where the learner recommends a best action for each context and is evaluated by context-weighted simple regret. Our guarantees are worst-case over the reward distributions, while remaining instance-dependent with respect to the context distribution vector $p$. Akin to experimental design problems where the population of interest is fixed but the sampled subpopulation can be controlled, we allow the learner to actively choose which context to sample from. For a known $p$, we characterize tight regret rates: passive sampling where contexts are randomly revealed achieves regret of order $\sqrt{n/T \, \lVert p \rVert_{1/2}}$, whereas active sampling with allocation $q_j \propto p_j^{2/3}$ achieves the tight rate $\sqrt{n/T} \, \lVert p \rVert_{2/3}$. The resulting improvement can be as large as $Θ(k^{1/4})$, where $k$ is the number of contexts. We further extend the analysis to budgeted active sampling, characterize the corresponding tight rate, and identify when a limited active budget suffices to recover the fully active rate. When $p$ is unknown, we propose the Explore-Explore-Then-Commit (EETC) algorithm, which optimally balances estimating the context distribution and the time to switch to active allocation, such that for large horizons, it matches the known-$p$ active rate up to constants. Experiments on synthetic and real-world data support our theoretical findings.

cs.LG↗

arXiv2026

Pure Exploration Beyond Reward Feedback: The Role of Post-Action Context

We introduce the problem of best arm identification (BAI) with post-action context, a new BAI problem in a stochastic multi-armed bandit environment and the fixed-confidence setting. The problem addresses the scenarios in which the learner receives a post-action context in addition to the reward after playing each action. This post-action context provides additional information that can significantly facilitate the decision process. We analyze two different types of the post-action context: (i) separator, where the reward depends solely on the context, and (ii) non-separator, where the reward depends on both the action and the context. For both cases, we derive instance-dependent lower bounds on the sample complexity and propose algorithms that asymptotically achieve the optimal sample complexity. For the separator setting, we propose a novel sampling rule called G-tracking, which uses the geometry of the context space to directly track the contexts rather than the actions. For the non-separator setting, we do so by demonstrating that the Track-and-Stop algorithm can be extended to this setting. Moreover, in both settings, we theoretically and empirically show that algorithms that ignore the post-action context are sub-optimal. Finally, our empirical results showcase the advantage of our approaches compared to the state of the art.

cs.LG↗

arXiv2026

Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets

We study optimistic bilevel optimization when the lower-level problem has a non-isolated manifold of minimizers. In this setting, the hyper-objective may be non-differentiable because the upper-level criterion must choose among multiple lower-level solutions. Under a local Polyak--Łojasiewicz (PŁ) condition, we show that differentiability does not require the lower-level solution set to be a singleton: uniqueness of the optimistic selection is sufficient. This yields an explicit pseudoinverse-based hyper-gradient formula extending the classical singleton-minimizer result. We further characterize the regularity of the hyper-objective: non-degeneracy of the selected minimizer along the solution manifold yields local smoothness, while failure of uniqueness can create many non-differentiable points and failure of non-degeneracy can destroy all positive Hölder regularity of the hyper-gradient. Motivated by this theory, we propose HG-MS, a select-then-differentiate method combining explicit optimistic selection with efficient pseudoinverse-based hyper-gradient computation. Despite the nonconvex nature of optimistic selection over the lower-level solution manifold, we show that HG-MS converges to a stationary point of the optimistic objective with complexity governed by the intrinsic dimension of the solution manifold rather than its ambient dimension. Empirically, we test a practical variant of HG-MS for matched-budget LLM source reweighting. This variant preserves the select-then-differentiate principle and obtains the best GSM8K/MATH scores across the tested backbones, along with competitive or best MT-Bench instruction-following results.

math.OC↗

arXiv2026

Inference Time Causal Probing in LLMs

Causal probing methods aim to test and control how internal representations influence the behavior of generative models. In causal probing, an intervention modifies hidden states so that a property takes on a different value. Most existing approaches define such interventions by training an auxiliary probe classifier, which ties the method to a specific task or model and risks misalignment with the model's predictive geometry. We propose Hidden-state Driven Margin Intervention (HDMI), a probe-free, gradient-based technique that directly steers hidden states using the model's native output. HDMI applies a margin objective that increases the probability of a target continuation while decreasing that of the source, without relying on probe classifiers. We further introduce a lookahead variant (LA-HDMI) for text editing that backpropagates through the softmax embeddings, modifying the current hidden state so that the likelihood of user-specified tokens increases in next token generations while preserving fluency. To evaluate interventions, we measure completeness (whether the targeted property changes as intended) and selectivity (whether unrelated properties are preserved), and report their harmonic mean as an overall measure of reliability. HDMI consistently achieves higher reliability than prior methods on the LGD agreement corpus and the CausalGym benchmark, across Meta-Llama-3-8B-Instruct, and Pythia-70M.

cs.AI↗

arXiv2026

Graph Learning Is Suboptimal in Causal Bandits

We study regret minimization in causal bandits under causal sufficiency where the underlying causal structure is not known to the agent. Previous work has focused on identifying the reward's parents and then applying classic bandit methods to them, or jointly learning the parents while minimizing regret. We investigate whether such strategies are optimal. Somewhat counterintuitively, our results show that learning the parent set is suboptimal. We do so by proving that there exist instances where regret minimization and parent identification are fundamentally conflicting objectives. We further analyze both the known and unknown parent set size regimes, establish novel regret lower bounds that capture the combinatorial structure of the action space. Building on these insights, we propose nearly optimal algorithms that bypass graph and parent recovery, demonstrating that parent identification is indeed unnecessary for regret minimization. Experiments confirm that there exists a large performance gap between our method and existing baselines in various environments.

cs.LG↗

arXiv2026

Fundamental Limits and Optimal Methods for Sharp Analytical Causal Bounds in Instrumental Variable Models

Bounding causal effects analytically, rather than numerically, is appealing for its interpretability and conceptual clarity. Existing sharp methods rely on optimization-based approaches such as the Balke-Pearl framework, whose computational complexity grows rapidly. An alternative line of work derives bounds heuristically using probability laws and generic inequalities, and some recent papers have claimed or conjectured that this approach can yield sharp analytical bounds with substantially lower complexity. In this paper, we show that this perceived advantage is illusory. In particular, in a discrete instrumental variable setting, we show that any sharp analytical bound for the average treatment effect must be expressible as a maximum (minimum) over a collection of linear terms whose cardinality grows exponentially in the number of values taken by the outcome. In parallel, we show that the number of instrumental variable inequalities itself also grows exponentially. Consequently, bounds and inequalities expressed using only polynomially many such terms cannot be sharp. As a constructive complement, the paper is accompanied by codes implemented in python and R to derive sharp analytical bounds and sharp inequalities with optimal efficiency, matching the lower bounds proven in this paper. These codes are available online.

stat.ME↗

arXiv2026

Causal Discovery in Linear Models with Unobserved Variables and Measurement Error

The presence of unobserved common causes and measurement error poses two major obstacles to causal structure learning, since ignoring either source of complexity can induce spurious causal relations among variables of interest. We study causal structure learning in linear systems where both challenges may occur simultaneously. We introduce a causal model called LV-SEM-ME, which contains four types of variables: directly observed variables, variables that are not directly observed but are measured with error, the corresponding measurements, and variables that are neither observed nor measured. Under a separability condition-namely, identifiability of the mixing matrix associated with the exogenous noise terms of the observed variables-together with certain faithfulness assumptions, we characterize the extent of identifiability and the corresponding observational equivalence classes. We provide graphical characterizations of these equivalence classes and develop recovery algorithms that enumerate all models in the equivalence class of the ground truth. We also establish, via a four-node union model that subsumes instrumental variable, front-door, and negative-control-outcome settings, a form of identification robustness: the target effect remains identifiable in the broader LV-SEM-ME model even when the assumptions underlying the specialized identification formulas for the corresponding submodels need not all hold simultaneously.

cs.LG↗

arXiv2026

Graph-Dependent Regret Bounds in Multi-Armed Bandits with Interference

We study multi-armed bandits under network interference, where each unit's reward depends on its own treatment and those of its neighbors in a given graph. This induces an exponentially large action space, making standard approaches computationally impractical. We propose a novel algorithm that uses the local graph structure to minimize regret. We derive a graph-dependent upper bound on cumulative regret that improves over prior work. Additionally, we provide the first lower bounds for bandits with arbitrary network interference, where each bound involves a distinct structural property of the graph. These bounds show that for both dense and sparse graphs, our algorithm is nearly optimal, with matching upper and lower bounds up to logarithmic factors. When the interference graph is unknown, a variant of our algorithm is Pareto optimal: no algorithm can uniformly outperform it across all instances. We complement our theoretical results with numerical experiments, showing that our approach outperforms the baseline methods.

cs.LG↗

arXiv2026

Zeroth-Order Stackelberg Control in Combinatorial Congestion Games

We study Stackelberg (leader--follower) tuning of network parameters (tolls, capacities, incentives) in combinatorial congestion games, where selfish users choose discrete routes (or other combinatorial strategies) and settle at a congestion equilibrium. The leader minimizes a system-level objective (e.g., total travel time) evaluated at equilibrium, but this objective is typically nonsmooth because the set of used strategies can change abruptly. We propose ZO-Stackelberg, which couples a projection-free Frank--Wolfe equilibrium solver with a zeroth-order outer update, avoiding differentiation through equilibria. We prove convergence to generalized Goldstein stationary points of the true equilibrium objective, with explicit dependence on the equilibrium approximation error, and analyze subsampled oracles: if an exact minimizer is sampled with probability $κ_m$, then the Frank--Wolfe error decays as $\mathcal{O}(1/(κ_m T))$. We also propose stratified sampling as a practical way to avoid a vanishing $κ_m$ when the strategies that matter most for the Wardrop equilibrium concentrate in a few dominant combinatorial classes (e.g., short paths). Experiments on real-world networks demonstrate that our method achieves orders-of-magnitude speedups over a differentiation-based baseline while converging to follower equilibria.

cs.GT↗

arXiv2026

Optimal Local Convergence Rates of Stochastic First-Order Methods under Local $α$-PL

We study the local convergence rate of stochastic first-order methods under a local $α$-Polyak-Lojasiewicz ($α$-PL) condition in a neighborhood of a target connected component $\mathcal{M}$ of the local minimizer set. The parameter $α\in [1,2]$ is the exponent of the gradient norm in the $α$-PL inequality: $α=2$ recovers the classical PL case, $α=1$ corresponds to Holder-type error bounds, and intermediate values interpolate between these regimes. Our performance criterion is the number of oracle queries required to output $\hat{x}$ with $F(\hat{x})-l \le \varepsilon$, where $l := F(y)$ for any $y \in \mathcal{M}$. We work in a local regime where the algorithm is initialized near $\mathcal{M}$ and, with high probability, its iterates remain in that neighborhood. We establish a lower bound $Ω(\varepsilon^{-2/α})$ for all stochastic first-order methods in this regime, and we obtain a matching upper bound $\mathcal{O}(\varepsilon^{-2/α})$ for $1 \le α< 2$ via a SARAH-type variance-reduced method with time-varying batch sizes and step sizes. In the convex setting, assuming a local $α$-PL condition on the $\varepsilon$-sublevel set, we further show a complexity lower bound $\widetildeΩ(\varepsilon^{-2/α})$ for reaching an $\varepsilon$-global optimum, matching the $\varepsilon$-dependence of known accelerated stochastic subgradient methods.

math.OC↗

arXiv2026

Neighborhood-Aware Graph Labeling Problem

Motivated by optimization oracles in bandits with network interference, we study the Neighborhood-Aware Graph Labeling (NAGL) problem. Given a graph $G = (V,E)$, a label set of size $L$, and local reward functions $f_v$ accessed via evaluation oracles, the objective is to assign labels to maximize $\sum_{v \in V} f_v(x_{N[v]})$, where each term depends on the closed neighborhood of $v$. Two vertices co-occur in some neighborhood term exactly when their distance in $G$ is at most $2$, so the dependency graph is the squared graph $G^2$ and $\mathrm{tw}(G^2)$ governs exact algorithms and matching fine-grained lower bounds. Accordingly, we show that this dependence is inherent: NAGL is NP-hard even on star graphs with binary labels and, assuming SETH, admits no $(L-\varepsilon)^{\mathrm{tw}(G^2)}\cdot n^{O(1)}$-time algorithm for any $\varepsilon>0$. We match this with an exact dynamic program on a tree decomposition of $G^2$ running in $O\!\left(n\cdot \mathrm{tw}(G^2)\cdot L^{\mathrm{tw}(G^2)+1}\right)$ time. For approximation, unless $\mathsf{P}=\mathsf{NP}$, for every $\varepsilon>0$ there is no polynomial-time $n^{1-\varepsilon}$-approximation on general graphs even under the promise $\mathrm{OPT}>0$; without the promise $\mathrm{OPT}>0$, no finite multiplicative approximation ratio is possible. In the nonnegative-reward regime, we give polynomial-time approximation algorithms for NAGL in two settings: (i) given a proper $q$-coloring of $G^2$, we obtain a $1/q$-approximation; and (ii) on planar graphs of bounded maximum degree, we develop a Baker-type polynomial-time approximation scheme (PTAS), which becomes an efficient PTAS (EPTAS) when $L$ is constant.

cs.DS↗

arXiv2025

Near-Optimal Experiment Design in Linear non-Gaussian Cyclic Models

We study the problem of causal structure learning from a combination of observational and interventional data generated by a linear non-Gaussian structural equation model that might contain cycles. Recent results show that using mere observational data identifies the causal graph only up to a permutation-equivalence class. We obtain a combinatorial characterization of this class by showing that each graph in an equivalence class corresponds to a perfect matching in a bipartite graph. This bipartite representation allows us to analyze how interventions modify or constrain the matchings. Specifically, we show that each atomic intervention reveals one edge of the true matching and eliminates all incompatible causal graphs. Consequently, we formalize the optimal experiment design task as an adaptive stochastic optimization problem over the set of equivalence classes with a natural reward function that quantifies how many graphs are eliminated from the equivalence class by an intervention. We show that this reward function is adaptive submodular and provide a greedy policy with a provable near-optimal performance guarantee. A key technical challenge is to efficiently estimate the reward function without having to explicitly enumerate all the graphs in the equivalence class. We propose a sampling-based estimator using random matchings and analyze its bias and concentration behavior. Our simulation results show that performing a small number of interventions guided by our stochastic optimization framework recovers the true underlying causal structure.

stat.ML↗

arXiv2025

Efficiently Escaping Saddle Points for Policy Optimization

Policy gradient (PG) is widely used in reinforcement learning due to its scalability and good performance. In recent years, several variance-reduced PG methods have been proposed with a theoretical guarantee of converging to an approximate first-order stationary point (FOSP) with the sample complexity of $O(ε^{-3})$. However, FOSPs could be bad local optima or saddle points. Moreover, these algorithms often use importance sampling (IS) weights which could impair the statistical effectiveness of variance reduction. In this paper, we propose a variance-reduced second-order method that uses second-order information in the form of Hessian vector products (HVP) and converges to an approximate second-order stationary point (SOSP) with sample complexity of $\tilde{O}(ε^{-3})$. This rate improves the best-known sample complexity for achieving approximate SOSPs by a factor of $O(ε^{-0.5})$. Moreover, the proposed variance reduction technique bypasses IS weights by using HVP terms. Our experimental results show that the proposed algorithm outperforms the state of the art and is more robust to changes in random seeds.

cs.LG↗

arXiv2025

Semiparametric Triple Difference Estimators

The triple difference causal inference framework is an extension of the well-known difference-in-differences framework. It relaxes the parallel trends assumption of the difference-in-differences framework through leveraging data from an auxiliary domain. Despite being commonly applied in empirical research, the triple difference framework has received relatively limited attention in the statistics literature. Specifically, investigating the intricacies of identification and the design of robust and efficient estimators for this framework has remained largely unexplored. This work aims to address these gaps in the literature. From the identification standpoint, we present outcome regression and weighting methods to identify the average treatment effect on the treated in both panel data and repeated cross-section settings. For the latter, we relax the commonly made assumption of time-invariant composition of units. From the estimation perspective, we develop semiparametric estimators for the triple difference framework in both panel data and repeated cross-sections settings. These estimators are based on the cross-fitting technique, and flexible machine learning tools can be used to estimate the nuisance components. We characterize conditions under which our proposed estimators are efficient, doubly robust, root-n consistent and asymptotically normal. As an application of our proposed methodology, we examined the effect of mandated maternity benefits on the hourly wages of women of childbearing age and found that these mandates result in a 2.6% drop in hourly wages.

econ.EM↗

arXiv2025

Fusing Rewards and Preferences in Reinforcement Learning

We present Dual-Feedback Actor (DFA), a reinforcement learning algorithm that fuses both individual rewards and pairwise preferences (if available) into a single update rule. DFA uses the policy's log-probabilities directly to model the preference probability, avoiding a separate reward-modeling step. Preferences can be provided by human-annotators (at state-level or trajectory-level) or be synthesized online from Q-values stored in an off-policy replay buffer. Under a Bradley-Terry model, we prove that minimizing DFA's preference loss recovers the entropy-regularized Soft Actor-Critic (SAC) policy. Our simulation results show that DFA trained on generated preferences matches or exceeds SAC on six control environments and demonstrates a more stable training process. With only a semi-synthetic preference dataset under Bradley-Terry model, our algorithm outperforms reward-modeling reinforcement learning from human feedback (RLHF) baselines in a stochastic GridWorld and approaches the performance of an oracle with true rewards.

cs.LG↗

arXiv2025

Hierarchical Reinforcement Learning with Targeted Causal Interventions

Hierarchical reinforcement learning (HRL) improves the efficiency of long-horizon reinforcement-learning tasks with sparse rewards by decomposing the task into a hierarchy of subgoals. The main challenge of HRL is efficient discovery of the hierarchical structure among subgoals and utilizing this structure to achieve the final goal. We address this challenge by modeling the subgoal structure as a causal graph and propose a causal discovery algorithm to learn it. Additionally, rather than intervening on the subgoals at random during exploration, we harness the discovered causal model to prioritize subgoal interventions based on their importance in attaining the final goal. These targeted interventions result in a significantly more efficient policy in terms of the training cost. Unlike previous work on causal HRL, which lacked theoretical analysis, we provide a formal analysis of the problem. Specifically, for tree structures and, for a variant of Erdős-Rényi random graphs, our approach results in remarkable improvements. Our experimental results on HRL tasks also illustrate that our proposed framework outperforms existing work in terms of training cost.

cs.LG↗