arXiv subjects

Xingwu Chen

Publications and source records attributed to Xingwu Chen.

At least 19 recordsLinked to original sources

arXiv2026

VALG: An Agentic System for ML Theory Research

Machine learning theory studies learning procedures through mathematical setups in which the data model, training protocol, oracle access, loss, metric, and randomness define the phenomenon that a theorem is meant to explain. Solving an open problem therefore requires the problem formulation, theorem target, and proof mechanism to be developed in concert. Researchers formulate hypotheses, test them through preliminary theoretical or empirical analysis, and refine both assumptions and proofs. We investigate whether this process can be organized as an autonomous agentic workflow for ML theory research. We develop VALG, an agentic system that combines multi-level Verification, Adaptive formulation of Learning-theory problems, and Graph-structured proof development. Within each source-relative theorem branch, VALG maintains a fixed mathematical specification, checks the theorem-level composition of a typed proof-dependency graph, and constructs and reviews local proofs in dependency order. When a proof attempt fails, VALG identifies whether the obstruction lies in a derivation, the proof structure, or the theorem formulation and routes the next attempt accordingly. Formulation-level obstructions initiate an explicitly related variant or relaxation, preserving the mathematical relation between the resulting theorem and the source problem. We evaluate VALG on nine subproblems from five COLT 2026 open problems. Two runs produce internally finalized theorem candidates that match the scope of their source briefs; the remaining seven yield restricted-method results, special cases, or conditional theorems. These case studies show how VALG keeps source-scope matches, relaxations, conditional results, and blocked attempts mathematically distinct. VALG is open source at https://github.com/DechenZhang/VALG-ML-Theory-Agent.

cs.AI↗

arXiv2026

Normal form method of center-focus problem in piecewise-smooth systems and algorithm design

We investigate the normal form method in piecewise-smooth monodromic systems to distinguish center from focus and determine the order of focus as well as degenerate Hopf bifurcations. For the normal form of piecewise-smooth monodromic systems given in previous publications in the sense of topologically equivalence, we prove the algebraic equivalence between the Lyapunov constants of the original piecewise-smooth system and the coefficient series of its normal form. With this algebraic equivalence, we prefer to compute the normal form coefficient series to avoid cumbersome integrals of trigonometric functions in the computation of Lyapunov constants, and find the relations of foci of the piecewise-smooth system and its subsystems: among the orders and among the stabilities. Moreover, we design algorithms for computing the normal form coefficient series and combine them with the obtained stability relation to construct stabilizing switching signals for state-dependent switched nonlinear systems, one basic problem in the control theory.

math.DS↗

arXiv2026

Bifurcations of grazing loops of arbitrary tangent multiplicity in piecewise-smooth systems

In piecewise-smooth differential systems, a hyperbolic limit cycle of a subsystem loses its structural stability if it grazes the switching manifold at a tangent point. Such a cycle is called a grazing loop and in this paper we investigate its bifurcations for arbitrary tangent multiplicity. For the low-multiplicity tangency, the recurrences are comprehensively captured by a functional perturbation with two parameters in previous publications, where the parameters characterize the recurrences near the tangent point and the limit cycle respectively. However, for high-multiplicity tangency, these parameters fail to capture the recurrences and thus, Poincare return maps can not be defined as usual. To address these challenges, we construct a functional perturbation with functions to clarify the recurrences and simultaneously, propose a localization method to make these two recurrences equivalent. We finally establish a quantitative relationship between the multiplicity of tangency and the numbers of crossing limit cycles, sliding loops bifurcating from the grazing loop and the number of tangent points on these sliding loops.

math.DS↗

arXiv2026

Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction

While LLMs demonstrate strong reasoning capabilities when provided with full information in a single turn, they exhibit substantial vulnerability in multi-turn interactions. Specifically, when information is revealed incrementally or requires updates, models frequently fail to integrate new constraints, leading to a collapse in performance compared to their single-turn baselines. We term the root cause as \emph{Contextual Inertia}: a phenomenon where models rigidly adhere to previous reasoning traces. Even when users explicitly provide corrections or new data in later turns, the model ignores them, preferring to maintain consistency with its previous (incorrect) reasoning path. To address this, we introduce \textbf{R}einforcement \textbf{L}earning with \textbf{S}ingle-\textbf{T}urn \textbf{A}nchors (\textbf{RLSTA}), a generalizable training approach designed to stabilize multi-turn interaction across diverse scenarios and domains. RLSTA leverages the model's superior single-turn capabilities as stable internal anchors to provide reward signals. By aligning multi-turn responses with these anchors, RLSTA empowers models to break contextual inertia and self-calibrate their reasoning based on the latest information. Experiments show that RLSTA significantly outperforms standard fine-tuning and abstention-based methods. Notably, our method exhibits strong cross-domain generalization (e.g., math to code) and proves effective even without external verifiers, highlighting its potential for general-domain applications. Code is available at https://github.com/Tencent/RLSTA.

cs.AI↗

arXiv2025

Crossing-sliding bifurcations in planar $\mathbb{Z}_2$-symmetric Filippov systems

In this paper we investigate the crossing-sliding bifurcations of planar Filippov systems with $\mathbb{Z}_2$-symmetry. Such bifurcations are triggered by the perturbations of a critical crossing cycle and constitute an important class of discontinuity-induced bifurcations. By constructing transition maps and developing a decomposition theorem of functions to overcome the difficulty of describing bifurcation boundaries in multi-parameter settings, we systematically characterize the codimension-one and codimension-two bifurcation scenarios through the explicit statement of non-degenerate conditions and the presentation of the corresponding bifurcation diagrams. The asymptotic properties of all bifurcation curves are also derived.

math.DS↗

arXiv2025

Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression

Using more test-time computation during language model inference, such as generating more intermediate thoughts or sampling multiple candidate answers, has proven effective in significantly improving model performance. This paper takes an initial step toward bridging the gap between practical language model inference and theoretical transformer analysis by incorporating randomness and sampling. We focus on in-context linear regression with continuous/binary coefficients, where our framework simulates language model decoding through noise injection and binary coefficient sampling. Through this framework, we provide detailed analyses of widely adopted inference techniques. Supported by empirical results, our theoretical framework and analysis demonstrate the potential for offering new insights into understanding inference behaviors in real-world language models.

cs.LG↗

arXiv2025

Normal forms of piecewise-smooth systems with a monodromic singular point

Normal form theory is developed deeply for planar smooth systems but has few results for piecewise-smooth systems because difficulties arise from continuity of the near-identity transformation, which is constructed piecewise. In this paper, we overcome the difficulties to study normal forms for piecewise-smooth systems with FF, FP, or PP equilibrium and obtain explicit any-order normal forms by finding piecewise-analytic homeomorphisms and deriving a new normal form for analytic systems. Our theorems of normal forms not only generalize previous results from second-order to any-order, from FF type to all FF, FP, PP types, but also provide a new method to compute Lyapunov constants, which are applied to solve the center problem and any-order Hopf bifurcations of piecewise-smooth systems.

math.DS↗

arXiv2025

Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection

While reinforcement learning (RL) demonstrated remarkable success in enhancing the reasoning capabilities of language models, the training dynamics of RL in LLMs remain unclear. In this work, we provide an explanation of the RL training process through empirical analysis and rigorous theoretical modeling. First, through systematic reasoning-pattern-level and token-level analysis across the RL training process, we show that while different reasoning patterns exhibit relatively stable success rates during training, RL primarily optimizes a sparse subset of critical tokens, thereby reshaping reasoning pattern distributions to affect model performance. Building on these empirical insights, we develop a theoretical framework to understand the training dynamics of RL with two typical rewards: verifiable reward (RLVR) and model's internal feedback (RLIF). For RLVR, we analyze the training dynamics under two special cases: one where models readily converge to optimal reasoning strategies, and another where optimization becomes challenging, revealing that the base model's reasoning quality is crucial for determining convergence behavior. For RLIF, we examine how internal rewards initially improve model performance but can potentially lead to degradation with continued training. Extensive experiments validate our findings, advancing both theoretical understanding and practical applications of RL in language model enhancement.

cs.LG↗

arXiv2025

Grazing-sliding bifurcations in planar $\mathbb{Z}_2$-symmetric Filippov systems

This paper aims to explore the effect of $\mathbb{Z}_2$-symmetry on grazing-sliding bifurcations in planar Filippov systems. We consider the scenario where the unperturbed system is $\mathbb{Z}_2$-symmetric and its subsystem exhibits a hyperbolic limit cycle grazing the discontinuity boundary at a fold. Employing differential manifold theory, we reveal the intrinsic quantities of unfolding all bifurcations and rigorously demonstrate the emergence of a codimension-two bifurcation under generic $\mathbb{Z}_2$-symmetric perturbations within the Filippov framework. After deriving an explicit non-degenerate condition with respect to parameters, we systematically establish the complete bifurcation diagram with exact asymptotics for all bifurcation boundaries by displacement map method combined with asymptotic analysis.

math.DS↗

arXiv2025

On the Robustness of Transformers against Context Hijacking for Linear Classification

Transformer-based Large Language Models (LLMs) have demonstrated powerful in-context learning capabilities. However, their predictions can be disrupted by factually correct context, a phenomenon known as context hijacking, revealing a significant robustness issue. To understand this phenomenon theoretically, we explore an in-context linear classification problem based on recent advances in linear transformers. In our setup, context tokens are designed as factually correct query-answer pairs, where the queries are similar to the final query but have opposite labels. Then, we develop a general theoretical analysis on the robustness of the linear transformers, which is formulated as a function of the model depth, training context lengths, and number of hijacking context tokens. A key finding is that a well-trained deeper transformer can achieve higher robustness, which aligns with empirical observations. We show that this improvement arises because deeper layers enable more fine-grained optimization steps, effectively mitigating interference from context hijacking. This is also well supported by our numerical experiments. Our findings provide theoretical insights into the benefits of deeper architectures and contribute to enhancing the understanding of transformer architectures.

cs.CL↗

arXiv2024

A quintic Z2-equivariant Li\'enard system arising from the complex Ginzburg-Landau equation: (II)

We continue to study a quintic Z2-equivariant Li\'enard system $\dot x=y,\dot y=-(a_0x+a_1x^3+a_2x^5)-(b_0+b_1x^2)y$ with $a_2b_1\ne 0$, arising from the complex Ginzburg-Landau equation. Global dynamics of the system have been studied in [{\it SIAM J. Math. Anal.}, {\bf 55}(2023) 5993-6038] when the sum of the indices of all equilibria is $-1$, i.e., $a_2<0$. The aim of this paper is to study the global dynamics of this quintic Li\'enard system when the sum of the indices of all equilibria is $1$, i.e., $a_2>0$.

math.CA↗

arXiv2024

How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression

Despite the remarkable success of transformer-based models in various real-world tasks, their underlying mechanisms remain poorly understood. Recent studies have suggested that transformers can implement gradient descent as an in-context learner for linear regression problems and have developed various theoretical analyses accordingly. However, these works mostly focus on the expressive power of transformers by designing specific parameter constructions, lacking a comprehensive understanding of their inherent working mechanisms post-training. In this study, we consider a sparse linear regression problem and investigate how a trained multi-head transformer performs in-context learning. We experimentally discover that the utilization of multi-heads exhibits different patterns across layers: multiple heads are utilized and essential in the first layer, while usually only a single head is sufficient for subsequent layers. We provide a theoretical explanation for this observation: the first layer preprocesses the context data, and the following layers execute simple optimization steps based on the preprocessed context. Moreover, we demonstrate that such a preprocess-then-optimize algorithm can significantly outperform naive gradient descent and ridge regression algorithms. Further experimental results support our explanations. Our findings offer insights into the benefits of multi-head attention and contribute to understanding the more intricate mechanisms hidden within trained transformers.

cs.LG↗

arXiv2024

Classifications and bifurcations of tangent points and their loops of planar piecewise-smooth systems

Tangent points, especial dynamics existing only in piecewise-smooth systems, usually have dynamical properties like equilibria of smooth systems. Loops connecting tangent points own partly properties of limit cycles and homoclinic loops of smooth systems. In this paper we give classifications for tangent points by tangency degree and for loops connecting them by configuration, and investigate their bifurcations. The classic method is to construct functional parameters for the case of low tangency degree but, is no longer valid for the case of general tangency degree, which leads to complicated interlacement of sliding and crossing motions on the switching manifold. We provide an explicit unfolding for tangent points of general tangency degree and their loops, in which explicit functional functions are constructed to replace functional parameters. We mainly obtain relations between original tangency degree and numbers of bifurcating tangent points, bifurcating tangent orbits and bifurcating loops for this unfolding. Some of these relations are generalizations to general tangency degree and others are new for previous publications.

math.DS↗

arXiv2024

Bifurcations and explicit unfoldings of grazing loops connecting one high multiplicity tangent point

For piecewise-smooth differential systems, in this paper we focus on crossing limit cycles and sliding loops bifurcating from a grazing loop connecting one high multiplicity tangent point. For the low multiplicity cases considered in previous publications, the method is to define and analyze return maps following the classic idea of Poincar\'e. However, high multiplicity leads to that either domains or properties of return maps are unclear under perturbations. To overcome these difficulties, we unfold grazing loops by functional parameters and functional functions, and analyze this unfolding along some specific parameter curve. Relationships between multiplicity and the numbers of crossing limit cycles and sliding loops are given, and our results not only generalize the results obtained in [J. Differential Equations 255(2013), 4403-4436; 269(2020), 11396-11434], but also are new for some specific grazing loops.

math.DS↗

arXiv2024

What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks

We study the capabilities of the transformer architecture with varying depth. Specifically, we designed a novel set of sequence learning tasks to systematically evaluate and comprehend how the depth of transformer affects its ability to perform memorization, reasoning, generalization, and contextual generalization. We show a transformer with only one attention layer can excel in memorization but falls short in other tasks. Then, we show that exhibiting reasoning and generalization ability requires the transformer to have at least two attention layers, while context generalization ability may necessitate three attention layers. Additionally, we identify a class of simple operations that a single attention layer can execute, and show that the complex tasks can be approached as the combinations of these simple operations and thus can be resolved by stacking multiple attention layers. This sheds light on studying more practical and complex tasks beyond our design. Numerical experiments corroborate our theoretical findings.

cs.LG↗

arXiv2023

Orbits and tsectors in irregular exceptional directions of full-null degenerate singular point

Near full-null degenerate singular points of analytic vector fields, asymptotic behaviors of orbits are not given by eigenvectors but totally decided by nonlinearities. Especially, in the case of high full-null degeneracy, i.e., the lowest degree of nonlinearities is high, such a singular point may have irregular exceptional directions and the blow-up technique can be hardly applied, which leaves a problem how to determine numbers of orbits and (elliptic, hyperbolic and parabolic) tangential sectors in this case. In this paper we work on this problem. Using Newton polygons to decompose nonlinearities into principal parts and remainder parts, we convert the problem to the numbers of nonzero real roots of edge-polynomials of principal parts. Computing Newton polygons for multiplication and differentiation of analytic functions and giving Newton polygons for addition, which was not found in literatures, we determine semi-definiteness of the Lie-bracket of principal parts and therefore obtain criteria for those numbers.

math.DS↗

arXiv2020

Dynamics of planar vector fields near a non-smooth equilibrium

In this paper we contribute to qualitative and geometric analysis of planar piecewise smooth vector fields, which consist of two smooth vector fields separated by the straight line $y=0$ and sharing the origin as a non-degenerate equilibrium. In the sense of $\Sigma$-equivalence, we provide a sufficient condition for linearization and give phase portraits and normal forms for these linearizable vector fields. This condition is hard to be weakened because there exist vector fields which are not linearizable when this condition is not satisfied. Regarding perturbations, a necessary and sufficient condition for local $\Sigma$-structural stability is established when the origin is still an equilibrium of both smooth vector fields under perturbations. In the opposition to this case, we prove that for any piecewise smooth vector field studied in this paper there is a limit cycle bifurcating from the origin, and there are some piecewise smooth vector fields such that for any positive integer $m$ there is a perturbation having exactly $m$ limit cycles bifurcating from the origin. Here $m$ maybe infinity.

math.DS↗

arXiv2019

Periodic orbits of linear Filippov systems with a line of discontinuity

In this paper we consider periodic orbits of planar linear Filippov systems with a line of discontinuity. Unlike many publications researching only the maximum number of crossing periodic orbits, we investigate not only the number and configuration of sliding periodic orbits, but also the coexistence of sliding periodic orbits and crossing ones. Firstly, we prove that the number of sliding periodic orbits is at most 2, and give all possible configurations of one or two sliding periodic orbits. Secondly, we prove that two sliding periodic orbits coexist with at most one crossing periodic orbit, and one sliding periodic orbit can coexist with two crossing ones.

math.DS↗