arXiv Science⌕ Search

SEARCH · arXiv Science

Search arXiv Science

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

111 recordsLinked to original sources
arXiv2026

Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes

We study gradient descent (GD) dynamics on logistic regression problems with large, constant step sizes. For linearly-separable data, it is known that GD converges to the minimizer with arbitrarily large step sizes, a property which no longer holds when the problem is not separable. In fact, the behaviour can be much more complex -- a sequence of period-doubling bifurcations begins at the critical step size $2/λ$, where $λ$ is the largest eigenvalue of the Hessian at the solution. Using a smaller-than-critical step size guarantees convergence if initialized nearby the solution: but does this suffice globally? In one dimension, we show that a step size less than $1/λ$ suffices for global convergence. However, for all step sizes between $1/λ$ and the critical step size $2/λ$, one can construct a dataset such that GD converges to a stable cycle. In higher dimensions, this is actually possible even for step sizes less than $1/λ$. Our results show that although local convergence is guaranteed for all step sizes less than the critical step size, global convergence is not, and GD may instead converge to a cycle depending on the initialization.

cs.LG↗
arXiv2026

Sample Complexity of Linear Quadratic Regulator Without Initial Stability

Inspired by REINFORCE, we introduce a novel receding-horizon algorithm for the Linear Quadratic Regulator (LQR) problem with unknown dynamics. Unlike prior methods, our algorithm avoids reliance on two-point gradient estimates while maintaining the same order of sample complexity. Furthermore, it eliminates the restrictive requirement of starting with a stable initial policy, broadening its applicability. Beyond these improvements, we introduce a refined analysis of error propagation through the contraction of the Riccati operator under the Riemannian distance. This refinement leads to a better sample complexity and ensures improved convergence guarantees.

math.OC↗
arXiv2026

An Energy-Based Mechanism for Compositional Behavior

Flexible intelligence relies on the ability to reuse previously acquired behaviors and combine them differently as circumstances change. In biological and artificial systems, this ability is often attributed to gating mechanisms that determine how much each available behavior should contribute at a given time. Yet these gating rules, the dynamics that compute them, and the neural circuits that may implement them are usually introduced separately, leaving unclear whether they reflect a common underlying principle. Here, we show that they can all be derived from a single variational principle for behavioral composition. The resulting mechanism naturally gives rise to softmax gating, evolves as an energy-based dynamical system with guaranteed convergence, and admits a recurrent neural network instantiation featuring context-dependent and local interactions. Across collective behavior, human decision-making, and layered control, the same mechanism reproduces characteristic behavioral patterns, provides interpretable accounts of how different behaviors are combined, and matches or outperforms established approaches. These results provide a unified account of how behavioral composition can emerge from a common principle, with implications for understanding flexible behavior in natural systems and for designing artificial agents that can adapt by recombining existing capabilities.

math.OC↗
arXiv2026

Constrained dynamics for searching saddle points on embedded Riemannian submanifolds of Euclidean space

Finding constrained saddle points on embedded Riemannian submanifolds of Euclidean space is significant for analyzing energy landscapes arising in physics and chemistry. Existing works exploit explicit global/local regular level-set representations of manifolds, which may be unavailable or computationally inconvenient for manifolds represented through, e.g., projectors, factorizations, or rank constraints. In this paper, we develop a constrained saddle dynamic based on embedded-submanifold geometric primitives, completely avoiding the use of explicit representations. In particular, our dynamic is formulated compactly on the Grassmann bundle of the tangent bundle. By analyzing the Grassmann bundle geometry, we rigorously establish the local linear stability of the dynamic and the local linear convergence of the resulting algorithms. Remarkably, our analysis provides the first iterate convergence result for discretized algorithms to saddle points of prescribed indices in embedded-submanifold settings. Moreover, by virtue of the Grassmann bundle formulation, we remove unnecessary nondegeneracy assumptions on the eigenvalues of the Riemannian Hessian that are present in existing works. We also point out that locating saddle points can be more ill-conditioned than finding local minimizers, and requires using nonredundant parametrizations. Finally, numerical experiments on linear eigenvalue problems and electronic excited-state calculations showcase the effectiveness of the proposed algorithms and corroborate the established local theory.

math.NA↗
arXiv2026

Scalable Self-Supervised Learning for Multiphase AC-OPF in Distribution Systems with Topology Reconfiguration

The proliferation of distributed energy resources (DERs) in distribution grids enables the active coordination of these assets to reduce costs and enable cleaner operations. Realizing this potential requires solving multiphase AC optimal power flow (AC-OPF) quickly across varying loads, DER availabilities, and topology reconfigurations, at much greater speed and scale than conventional nonlinear solvers. Learning-based surrogates can offer millisecond inference, yet existing methods target largely balanced transmission systems and do not scale to the multiphase, unbalanced, and reconfigurable nature of distribution feeders at utility scale. We present the Penalty + Sequential Linearized Feasibility Seeking (SLFS) algorithm, a self-supervised learning framework for multiphase distribution AC-OPF under switch-induced topology changes. Penalty+SLFS requires no labeled optimal solutions and trains directly from the AC-OPF objective and constraints through a differentiable fixed-point power flow solver, avoiding expensive label generation and admitting robust training procedures. Topology changes are handled efficiently using Sherman-Morrison-Woodbury updates of the admittance-matrix inverse, while an M-step Jacobian approximation accelerates differentiation through the power flow solver. At inference, SLFS repairs any infeasible predictions, providing feasibility guarantees with low computational overhead. On IEEE feeders ranging from 13 to 8,500 nodes, Penalty+SLFS achieves negligible optimality gaps and near-zero constraint violations, delivers up to three orders of magnitude speedups over IPOPT, and remains robust under large distributional shifts, demonstrating a viable path toward real-time, topology-aware AC-OPF for large-scale distribution grids.

eess.SY↗
arXiv2026

Turnpike properties in nonlinear system identification

We analyze the problem of learning general discrete-time nonlinear state-space models using the simulation error minimization (SEM) method. In this setting, model parameters are typically learned by minimizing the mismatch between simulated and measured outputs over a training dataset, or shorter subsequences extracted from it. Specifically, we study the cumulative output turnpike property of the underlying SEM optimization problem, which requires optimal output sequences emanating from a fixed initial state to approach and remain close to an optimal output sequence of the corresponding SEM problem with free initial state. In the presence of non-unique optimal output sequences---as may arise, for instance, in fully black-box system identification using neural networks---the property is formulated with respect to the closest such sequence. Turnpike behavior is generally desirable in practice, as it provides a theoretical justification for employing computationally more tractable SEM formulations with fixed initial states while ensuring that their optimal output sequences remain close to unconstrained optimal ones. Under a mild reachability condition, we establish equivalence between the cumulative turnpike property, coercivity of the value function, and a tailored notion of strict dissipativity. We additionally introduce a cardinality turnpike property and show that it is strictly weaker than the cumulative notion. Finally, we establish sufficient conditions for turnpike behavior based on incremental output stability, convexity of the stage cost, and a suitable optimality condition, and illustrate the theory by means of a numerical example.

eess.SY↗
arXiv2026

Two Adjoint Perspectives on Fokker-Planck Optimization: A Microscopic-Macroscopic Correspondence

The Fokker-Planck equation admits both a macroscopic Eulerian description through probability densities and a microscopic Lagrangian description through stochastic trajectories. Consequently, optimization problems constrained by the Fokker-Planck equation can be formulated from either perspective. Surprisingly, the corresponding adjoint equations appear to be fundamentally different: the macroscopic adjoint is governed by the backward Kolmogorov equation, whereas the microscopic adjoint evolves pathwise along stochastic trajectories. In this note, we reconcile these two formulations by establishing their correspondence in the continuum setting. We further show that, although their discrete gradients no longer coincide after discretization, both provide consistent numerical approximations of the continuum gradient. Explicit convergence rates are established for both discretization strategies.

math.NA↗
arXiv2026

Characterizations and Complexity of Minimum Forward and Integer Cycle Bases

The cycle space of a directed graph is generated by a cycle basis, where, in general, cycles are allowed to have both forward and backward arcs. In a forward cycle, all arcs must follow the given direction. Several open questions remain regarding the complexity of the minimum cycle basis problem, in particular the minimum-weight integral cycle basis problem, and the minimum-weight weakly and strictly fundamental forward cycle basis problems. In this paper, we address these open questions. First, we study the existence, structure, and computational complexity of minimum-weight forward cycle bases. We give a complete structural characterization of digraphs that admit weakly fundamental (and hence integral) forward cycle bases. We further provide a characterization when a strongly connected digraph admits a forward fundamental cycle basis, proving that such a basis exists if and only if the set of directed cycles has cardinality equal to the cycle rank; in this case, the basis is unique. Lastly, we show that while minimum-weight forward fundamental cycle bases can be found in polynomial time whenever they exist, the minimum-weight forward weakly fundamental cycle basis problem is APX-hard via an L-reduction from the minimum-weight weakly fundamental cycle basis problem on digraphs with metric weights. Second, we introduce opt-in graphs, i.e., the family of graphs for which minimum cycle bases are integral for any weight function. We show that this family is minor-closed and hence, by the Robertson-Seymour theorem, is characterized by a finite set of forbidden minors, so that the opt-in recognition problem is solvable in polynomial time. Lastly, we present an algorithm to check whether a graph is opt-in, and if not, to identify which of its minors belong to the set of forbidden minors. Applying this algorithm, we show that the complete graph $K_n$ is opt-in if and only if $n \leq 7$.

math.OC↗
arXiv2026

A simple derivation of the Kalman filter

In this lecture note, we present a concise and self-contained derivation of the discrete-time Kalman filter equations that requires only a basic understanding of least squares estimation. The treatment is designed to minimize mathematical overhead while preserving both rigor and generality.

math.OC↗
arXiv2026

On the invariance of risk-sensitive LQR gain under input randomization

This paper shows that the optimal gain of the risk-sensitive linear quadratic regulator (LQR) problem is invariant under input randomization, i.e., when the controller deliberately injects noise into the nominal control input. This appears counterintuitive at first glance because certainty equivalence does not hold for risk-sensitive LQR and input randomization inflates the effective process noise. Nonetheless, the gain is preserved because the input noise enters not only the system dynamics but also the cost functional, and its total effect on the gain eventually vanishes. Consequently, the optimal gain and its associated Riccati recursion need not be recomputed, and the increment in the optimal cost can be readily evaluated in closed form. This result facilitates the use of risk-sensitive LQR in applications that employ input randomization for privacy or exploration, such as watermarking for replay attack detection, differential privacy, and path integral control.

math.OC↗
arXiv2026

Generalized Regret Analysis of Thompson Sampling using Fractional Posteriors

Thompson sampling (TS) is one of the most popular and earliest algorithms to solve stochastic multi-armed bandit problems. We consider a variant of TS, named $α$-TS, where we use a fractional or $α$-posterior ($α\in(0,1)$) instead of the standard posterior distribution. To compute an $α$-posterior, the likelihood in the definition of the standard posterior is tempered with a factor $α$. For $α$-TS we obtain both instance-dependent $\mathcal{O}\left(\sum_{k \neq i^*} Δ_k\left(\frac{\log(T)}{C(α)Δ_k^2} + \frac{1}{2} \right)\right)$ and instance-independent $\mathcal{O}(\sqrt{KT\log K})$ frequentist regret bounds under very mild conditions on the prior and reward distributions, where $Δ_k$ is the gap between the true mean rewards of the $k^{th}$ and the best arms, and $C(α)$ is a known constant. Both the sub-Gaussian and exponential family models satisfy our general conditions on the reward distribution. Our conditions on the prior distribution can be easily satisfied by a density that is positive, continuous, and bounded. We also establish another instance-dependent regret upper bound that matches (up to constants) to that of improved UCB [Auer and Ortner, 2010]. Our regret analysis carefully adapts and combines recent theoretical developments in the non-asymptotic concentration analysis and Bernstein-von Mises type results for the $α$-posterior distribution. Moreover, our analysis does not require additional structural properties such as closed-form posteriors or conjugate priors.

stat.ML↗
arXiv2026

A Feedback Linearized Model Predictive Control Strategy for Input-Constrained Self-Driving Cars

This paper proposes a novel real-time affordable solution to the trajectory tracking control problem for self-driving cars subject to longitudinal and steering angular velocity constraints. To this end, we develop a dual-mode Model Predictive Control (MPC) solution starting from an input-output feedback linearized description of the vehicle kinematics. First, we derive the state-dependent input constraints acting on the linearized model and characterize their worst-case time-invariant inner approximation. Then, a dual-mode MPC is derived to be real-time affordable and ensuring, by design, constraints fulfillment, recursive feasibility, and uniformly ultimate boundedness of the tracking error in an ad-hoc built robust control invariant region. The approach's effectiveness and performance are experimentally validated via laboratory experiments on a Quanser Qcar. The obtained results show that the proposed solution is computationally affordable and with tracking capabilities that outperform two alternative control schemes.

eess.SY↗
arXiv2026

Infrequent Resolving Algorithm for Online Linear Programming

Online linear programming (OLP) has gained significant attention from both researchers and practitioners due to its extensive applications such as online auctions, network revenue management, order fulfillment and advertising. Existing OLP algorithms fall into two categories: LP-based algorithms and LP-free algorithms. The former typically guarantees better performance but requires solving a large number of LPs, which could be computationally expensive. In contrast, LP-free algorithms only require first-order computations but induce a worse performance. In this work, we bridge the gap between these two extremes by proposing a well-performing algorithm that solves LPs at a few selected time points and conducts first-order computations at other time points. Specifically, for the case where the inputs are drawn from an unknown finite-support distribution, the proposed algorithm achieves a constant regret (even for the hard "degenerate" case) while solving LPs only $O(\log\log T)$ times over the time horizon $T$. Moreover, when we are allowed to solve LPs only $M$ times, we design the corresponding schedule such that the proposed algorithm can guarantee a nearly $O\left(T^{(1/2)^{M-1}}\right)$ regret. Our work highlights the value of resolving both at the beginning and the end of the selling horizon, and provides a novel framework to prove the performance guarantee of the proposed policy under different infrequent resolving schedules. Numerical experiments are conducted to demonstrate the efficiency of the proposed algorithms.

cs.DS↗
arXiv2026

A Compositional Kernel Model for Feature Learning

We study a compositional variant of kernel ridge regression in which the predictor is applied to a coordinate-wise reweighting of the inputs. Formulated as a variational problem, this model provides a tractable setting for studying feature learning in compositional architectures. From the perspective of variable selection, we show how relevant variables are recovered while noise variables are eliminated. We prove that both global minimizers and stationary points discard noise coordinates when the noise variables are Gaussian distributed. A central finding is that $\ell_1$-type kernels, such as the Laplace kernel, succeed in recovering features contributing to nonlinear effects at stationary points, whereas Gaussian kernels recover only linear ones.

cs.LG↗
arXiv2026

Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU

In this work, we study the nonlinear dynamics of a shallow neural network trained with mean-squared loss and leaky ReLU activation. Under Gaussian inputs and equal layer width k, (1) we establish, based on the equivariant gradient degree, a theoretical framework, applicable to any number of neurons k>= 4, to detect bifurcation of critical points with associated symmetries from global minimum as leaky parameter $α$ varies. Typically, our analysis reveals that a multi-mode degeneracy consistently occurs at the critical number 0, independent of k. (2) As a by-product, we further show that such bifurcations are width-independent, arise only for nonnegative $α$ and that the global minimum undergoes no further symmetry-breaking instability throughout the engineering regime $α$ in range (0,1). An explicit example with k=5 is presented to illustrate the framework and exhibit the resulting bifurcation together with their symmetries.

math.OC↗
arXiv2026

Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method

The adaptive moment estimation (Adam) optimizer proposed by Kingma & Ba (2014) is presumably the most popular stochastic gradient descent (SGD) optimization method for the training of deep neural networks (DNNs) in artificial intelligence (AI) systems. Despite its groundbreaking success in the training of AI systems, it still remains an open research problem to provide a complete error analysis of Adam, not only for optimizing DNNs but even when applied to strongly convex stochastic optimization problems (SOPs). Previous error analysis results for strongly convex SOPs in the literature provide conditional convergence analyses that rely on the assumption that Adam does not diverge to infinity but remains uniformly bounded. It is the key contribution of this work to establish uniform a priori bounds for Adam and, thereby, to provide -- for the first time -- an unconditional error analysis for Adam for a large class of strongly convex SOPs.

cs.LG↗
arXiv2026

Adjoint DSMC Method for Spatially Inhomogeneous Boltzmann Equation with General Boundary Conditions

We develop adjoint Direct Simulation Monte Carlo (DSMC) formulations for the spatially inhomogeneous Boltzmann equation with periodic, specular reflecting, diffuse thermal, and prescribed inflow boundary conditions. Periodic and specular boundaries are treated using a pathwise particle adjoint conditional on the realized event history. For diffuse thermal boundaries, we introduce a randomized-time regularization of wall-crossing events and use score-function terms to differentiate the resulting boundary probabilities. Reparameterization of the outgoing half-Maxwellian samples provides sensitivities with respect to wall temperatures and tangential wall velocities. Prescribed inflow requires a different construction because perturbations of incoming particles affect subsequent cell populations, local collision frequencies, and collision schedules. We therefore derive an ensemble adjoint based on the locally linearized Boltzmann collision operator and evaluate boundary sensitivities using local inflow-source scores, with injection counts held fixed. For a scalar objective, the dominant adjoint cost is largely independent of the number of parameters. Numerical experiments for Maxwell molecules validate the formulations against centered finite differences for thermal, mixed thermal-specular, two-sided inflow, and high-Mach Couette-flow configurations. The results demonstrate consistent gradient estimates, Monte Carlo convergence, stability with respect to the thermal regularization parameter, and accurate sensitivity calculation in a regime with limited relative statistical noise.

math.NA↗
arXiv2026

The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting

An agent's probability report is paid for twice: by a strictly proper scoring rule, and by an approval rule for the decision it triggers. In this classical decision-coupled setting, non-affine approval is known to defeat truthful reporting. We show the conflict is endogenous: when feasible, the welfare-maximizing approval rule is never affine. The distortion, however, is predictable and can be designed around. There is a reserve report at which pretending to be the marginal type costs exactly the approval prize. Approving at or above the reserve screens types perfectly under every strictly proper score, and the reserve does not depend on the type distribution. A Lipschitz rule with a single kink attains first-best exactly; under strict feasibility no continuously differentiable rule does. The binding constraint is steepness, not smoothness. First-best is attainable within a slope budget if and only if the budget is at least the critical slope: the steepest chord of the pretending cost up to the reserve. Below it the welfare loss is cubic in the shortfall. Where the pretending cost is convex up to the reserve, as for Brier, log and power scores, the critical slope is closed-form. The instances are AI-agent oversight and marketplace operation.

cs.GT↗