arXiv subjects

Jingfan Zhang

Publications and source records attributed to Jingfan Zhang.

9 recordsLinked to original sources

arXiv2026

Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target

Reasoning post-training draws on offline demonstrations and policy-generated rollouts, yet conventional supervised fine-tuning (SFT) followed by reinforcement learning (RL) separates their objectives and limits continued reuse of collected responses. We propose \textbf{Cross-Behavior Policy Optimization} (CBPO), which makes these sources compatible with one regression loss through a fixed-reference target. Because the generating policy does not enter this target, positive and negative offline completions, reference responses, and current and historical rollouts can share a common reward-labeled record format and replay buffer. As new responses enter the buffer, optimization retains the same regression objective without coordinating separate SFT and RL objectives or requiring stored behavior-policy probabilities, importance weighting, or a learned critic. Restricting the available sources gives offline, on-policy, and off-policy variants, while CBPO-Unified combines them. Across two model scales and four mathematical reasoning benchmarks, CBPO delivers leading \texttt{avg@32} and \texttt{pass@32} performance across benchmarks and data regimes. On Qwen3-1.7B, CBPO-Unified improves benchmark-averaged \texttt{avg@32} and \texttt{pass@32} over SFT DAPO by 2.33 and 4.44 percentage points, respectively. In the replay ablation, retaining historical rollouts improves these metrics by 1.87 and 3.72 percentage points over the no-history configuration.

cs.LG↗

arXiv2026

OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search

The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We study whether tree search improves their coverage within a fixed budget while controlling gradient bias. We introduce On-Policy Parallel Tree Search (OPTS) and Tree Trajectory Policy Optimization (TTPO) using on-policy tree trajectories, which sample new suffixes from the current policy at visited states. This needs no action-distribution correction, although branching changes state visitation. Our Branch Aggregation Lemma shows that branch-weighted tree statistics recover chain expectations when branch choices and weights are fixed before outgoing transitions are sampled. OPTS selects expansion states using estimated performance differences. Under deterministic dynamics, exact values, and max-backup advantages, the induced search policy's expected return improves monotonically with the budget. We bound the gradient bias from adaptive expansion and show that max backup assigns prefix credit to actions leading to better discovered suffixes. Against a finite chain reference, TTPG's measured bias stays near its no-branching level, while NaivePG's bias grows from 0.1251 to 0.4884. At matched budgets, reward- and value-guided OPTS improve correct-answer coverage and majority-vote accuracy over independent sampling. At matched branch counts, OPTS + TTPG gains coverage with a modest bias increase relative to Fixed-branch + TTPG. Under matched interaction or rollout budgets, OPTS-TTPO improves MuJoCo tail returns over PPO by up to 28.6%, achieves a 34-22-1 win-loss-tie record against PPO on Atari-57 under the last-100-log mean-return metric, and improves micro-averaged avg@32 and pass@32 over PPO across all four Qwen3 models.

cs.CL↗

arXiv2025

Towards Human-level Intelligence via Human-like Whole-Body Manipulation

Building general-purpose intelligent robots has long been a fundamental goal of robotics. A promising approach is to mirror the evolutionary trajectory of humans: learning through continuous interaction with the environment, with early progress driven by the imitation of human behaviors. Achieving this goal presents three core challenges: (1) designing safe robotic hardware with human-level physical capabilities; (2) developing an intuitive and scalable whole-body teleoperation interface for data collection; and (3) creating algorithms capable of learning whole-body visuomotor policies from human demonstrations. To address these challenges in a unified framework, we propose Astribot Suite, a robot learning suite for whole-body manipulation aimed at general daily tasks across diverse environments. We demonstrate the effectiveness of our system on a wide range of activities that require whole-body coordination, extensive reachability, human-level dexterity, and agility. Our results show that Astribot's cohesive integration of embodiment, teleoperation interface, and learning pipeline marks a significant step towards real-world, general-purpose whole-body robotic manipulation, laying the groundwork for the next generation of intelligent robots.

cs.RO↗

arXiv2025

Simulink Mutation Testing using CodeBERT

We present BERTiMuS, an approach that uses CodeBERT to generate mutants for Simulink models. BERTiMuS converts Simulink models into textual representations, masks tokens from the derived text, and uses CodeBERT to predict the masked tokens. Simulink mutants are obtained by replacing the masked tokens with predictions from CodeBERT. We evaluate BERTiMuS using Simulink models from an industrial benchmark, and compare it with FIM -- a state-of-the-art mutation tool for Simulink. We show that, relying exclusively on CodeBERT, BERTiMuS can generate the block-based Simulink mutation patterns documented in the literature. Further, our results indicate that: (a) BERTiMuS is complementary to FIM, and (b) when one considers a requirements-aware notion of mutation testing, BERTiMuS outperforms FIM.

cs.SE↗

arXiv2024

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning

Low-rank adaptation (LoRA) and its mixture-of-experts (MOE) variants are highly effective parameter-efficient fine-tuning (PEFT) methods. However, they introduce significant latency in multi-tenant settings due to the LoRA modules and MOE routers added to multiple linear modules in the Transformer layer. To address this issue, we propose Mixture of Low-Rank Adaptation (MiLoRA), a novel and efficient LoRA variant. MiLoRA differs from previous MOE-style LoRA methods by considering each LoRA module as an expert and employing a prompt-aware routing mechanism. This mechanism calculates expert routing results once before generating the first new token and reuses these results for subsequent tokens, reducing latency. Extensive experiments and analysis on commonsense reasoning tasks, math reasoning tasks, and widely used LLM evaluation benchmarks demonstrate that MiLoRA consistently outperforms strong PEFT baselines with comparable tunable parameter budgets. Additionally, MiLoRA significantly reduces latency in multi-tenant settings compared to previous LoRA-based methods.

cs.CL↗

arXiv2021

Phase limitations of multipliers at harmonics

We present a phase condition under which there is no suitable multiplier for a given continuous-time plant. The condition can be derived from either the duality approach or from the frequency interval approach. The condition has a simple graphical interpretation, can be tested in a numerically efficient manner and may be applied systematically. Numerical examples show significant improvement over existing results in the literature. The condition is used to demonstrate a third order system with delay that is a counterexample to the Kalman Conjecture.

math.OC↗

arXiv2020

Duality bounds for discrete-time Zames-Falb multipliers

We develop phase limitations for the discrete-time Zames-Falb multipliers based on the separation theorem for Banach spaces. By contrast with their continuous-time counterparts they lead to numerically efficient results that can be computed either in closed form or via a linear program. The closed-form phase limitations are tight in the sense that we can construct multipliers that meet them with equality. We discuss numerical examples where the limitations are stronger than others in the literature. The numerical results complement searches for multipliers in the literature; they allow us to show, by construction, that the set of plants for which a suitable Zames-Falb multiplier exists is non-convex.

math.OC↗

arXiv2019

Noncausal FIR Zames-Falb Multiplier Search for Exponential Convergence Rate

In the existing literature, there are two approaches to estimate tighter bounds of the exponential convergence rate of stable Lure systems. On one hand, the classical integral quadratic constraint (IQC) framework can be applied under loop-transformation, so the stability of the new loop implies the convergence of the original loop. On the other hand, it is possible to modify the IQC framework, the so-called rho-IQC framework, in such a way that the convergence rate is directly obtained over the original loop. In this technical note, we extend the literature results from the search for a causal finite impulse response (FIR) Zames-Falb multiplier to the noncausal case. We show that the multipliers by the two approaches are equivalent by a change of variable. However, the factorisation of the Zames-Falb rho-IQC is restricted compared to the Zames-Falb IQC, so an unified factorisation is proposed. Finally, numerical examples illustrate that noncausal multipliers lead to less-conservative results.

math.OC↗

arXiv2018

Convex searches for discrete-time Zames-Falb multipliers

In this paper we develop and analyse convex searches for Zames--Falb multipliers. We present two different approaches: Infinite Impulse Response (IIR) and Finite Impulse Response (FIR) multipliers. The set of FIR multipliers is complete in that any IIR multipliers can be phase-substituted by an arbitrarily large order FIR multiplier. We show that searches in discrete-time for FIR multipliers are effective even for large orders. As expected, the numerical results provide the best $\ell_{2}$-stability results in the literature for slope-restricted nonlinearities. Finally, we demonstrate that the discrete-time search can provide an effective method to find suitable continuous-time multipliers.

math.OC↗