arXiv subjects

Tianyi Zhang

Publications and source records attributed to Tianyi Zhang.

At least 163 records · Page 9Linked to original sources

arXiv2025

Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation

Recent decoding methods improve the factuality of large language models (LLMs) by refining how the next token is selected during generation. These methods typically operate at the token level, leveraging internal representations to suppress superficial patterns. Nevertheless, LLMs remain prone to hallucinations, especially over longer contexts. In this paper, we propose Active Layer-Contrastive Decoding (ActLCD), a novel decoding strategy that actively decides when to apply contrasting layers during generation. By casting decoding as a sequential decision-making problem, ActLCD employs a reinforcement learning policy guided by a reward-aware classifier to optimize factuality beyond the token level. Our experiments demonstrate that ActLCD surpasses state-of-the-art methods across five benchmarks, showcasing its effectiveness in mitigating hallucinations in diverse generation scenarios.

cs.CL↗

arXiv2025

Unlock giant nonreciprocity via multi-valued behavior of non-Hermitian zero-index materials

Although Einstein's field equations are time-independent, the multivalued feature of the horizon of a blackhole naturally enables the one-way transmission, leading to the strong arrow of time from the time-independent gravitational interaction. Here we experimentally demonstrate a photonic analogue of this principle and reveal the infinite nonreciprocity of the time-reversal-symmetric Maxwell equations. By designing a non-Hermitian zero-index magneto-optical metawaveguide, we introduce multivalued feature to this metawaveguide's complex eigenspace via an exceptional point with non-zero residue, bringing nonlocal, path-dependent historical memory to the system. Hence, a weak magneto-optical response can direct forward and backward waves to two photonic branches with largely distinct momenta and losses, leading to the optical nonreciprocity far beyond the limitation imposed by the magneto-optical material. We fabricated an a-Si/Ce:YIG metawaveguide, achieving nonreciprocal phase shift of 47.78 rad/mm and nonreciprocal loss of 53.9 dB/mm near 1575 nm, exceeding state-of-the-art nonreciprocal devices by an order of magnitude. Our principle universally applies from microwave to visible frequencies, leading to compact isolators, circulators, and sensors. Our principle can also be extended to nonreciprocal acoustic, elastic, and thermal systems. The proposed new paradigm -- geometry-based strong arrow of time in covariant and reversible physical systems -- has broad implications in many disciplines including string theory, cosmology, and astronomy.

physics.optics↗

arXiv2025

Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy

While recent vision-language-action models trained on diverse robot datasets exhibit promising generalization capabilities with limited in-domain data, their reliance on compact action heads to predict discretized or continuous actions constrains adaptability to heterogeneous action spaces. We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences through a unified multimodal diffusion process. Departing from prior methods that condition denoising on fused embeddings via shallow networks, Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations. This design explicitly models action deltas and environmental nuances. By scaling the diffusion action denoiser alongside the Transformer's scalability, Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces. Such synergy enhances robustness against various variances and facilitates the successful execution of long-horizon tasks. Evaluations across extensive benchmarks demonstrate state-of-the-art or comparative performance in simulation. Notably, Dita achieves robust real-world adaptation to environmental variances and complex long-horizon tasks through 10-shot finetuning, using only third-person camera inputs. The architecture establishes a versatile, lightweight and open-source baseline for generalist robot policy learning. Project Page: https://robodita.github.io.

cs.RO↗

arXiv2025

InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

We introduce InternVL 3.5, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the Cascade Reinforcement Learning (Cascade RL) framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a Visual Resolution Router (ViR) that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled Vision-Language Deployment (DvD) strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0\% gain in overall reasoning performance and a 4.05$\times$ inference speedup compared to its predecessor, i.e., InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks -- narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released.

cs.CV↗

arXiv2025

Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models

Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as OpenAI o1 and DeepSeek-R1, have further improved performance in System-2 reasoning domains like mathematics and programming by harnessing supervised fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance the Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences improve performance, they also introduce significant computational overhead due to verbose and redundant outputs, known as the "overthinking phenomenon". In this paper, we provide the first structured survey to systematically investigate and explore the current progress toward achieving efficient reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we categorize existing works into several key directions: (1) model-based efficient reasoning, which considers optimizing full-length reasoning models into more concise reasoning models or directly training efficient reasoning models; (2) reasoning output-based efficient reasoning, which aims to dynamically reduce reasoning steps and length during inference; (3) input prompts-based efficient reasoning, which seeks to enhance reasoning efficiency based on input prompt properties such as difficulty or length control. Additionally, we introduce the use of efficient data for training reasoning models, explore the reasoning capabilities of small language models, and discuss evaluation methods and benchmarking. Project website: https://github.com/Eclipsess/Awesome-Efficient-Reasoning-LLMs

cs.CL↗

arXiv2025

Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy

Vision-Language-Action (VLA) models frequently encounter challenges in generalizing to real-world environments due to inherent discrepancies between observation and action spaces. Although training data are collected from diverse camera perspectives, the models typically predict end-effector poses within the robot base coordinate frame, resulting in spatial inconsistencies. To mitigate this limitation, we introduce the Observation-Centric VLA (OC-VLA) framework, which grounds action predictions directly in the camera observation space. Leveraging the camera's extrinsic calibration matrix, OC-VLA transforms end-effector poses from the robot base coordinate system into the camera coordinate system, thereby unifying prediction targets across heterogeneous viewpoints. This lightweight, plug-and-play strategy ensures robust alignment between perception and action, substantially improving model resilience to camera viewpoint variations. The proposed approach is readily compatible with existing VLA architectures, requiring no substantial modifications. Comprehensive evaluations on both simulated and real-world robotic manipulation tasks demonstrate that OC-VLA accelerates convergence, enhances task success rates, and improves cross-view generalization. The code will be publicly available.

cs.RO↗

arXiv2025

Covering the Euclidean Plane by a Pair of Trees

A {$t$-stretch tree cover} of a metric space $M = (X,δ)$, for a parameter $t \ge 1$, is a collection of trees such that every pair of points has a $t$-stretch path in one of the trees. Tree covers provide an important sketching tool that has found various applications over the years. The celebrated {Dumbbell Theorem} by Arya et al. [STOC'95] states that any set of points in the Euclidean plane admits a $(1+ε)$-stretch tree cover with $O_ε(1)$ trees. This result extends to any (constant) dimension and was also generalized for arbitrary doubling metrics by Bartal et al. [ICALP'19]. Although the number of trees provided by the Dumbbell Theorem is constant, this constant is not small, even for a stretch significantly larger than $1+ε$. At the other extreme, any single tree on the vertices of a regular $n$-polygon must incur a stretch of $Ω(n)$. Using known results of ultrametric embeddings, one can easily get a stretch of $\tilde{O}(\sqrt{n})$ using two trees. The question of whether a low stretch can be achieved using two trees has remained illusive, even in the Euclidean plane. In this work, we resolve this fundamental question in the affirmative by presenting a constant-stretch cover with a pair of trees, for any set of points in the Euclidean plane. Our main technical contribution is a {surprisingly simple} Steiner construction, for which we provide a {tight} stretch analysis of $\sqrt{26}$. The Steiner points can be easily pruned if one is willing to increase the stretch by a small constant. Moreover, we can bound the maximum degree of the construction by a constant. Our result thus provides a simple yet effective reduction tool -- for problems that concern approximate distances -- from the Euclidean plane to a pair of trees. To demonstrate the potential power of this tool, we present some applications [...]

cs.CG↗

arXiv2025

Qualitative Study for LLM-assisted Design Study Process: Strategies, Challenges, and Roles

Design studies aim to create visualization solutions for real-world problems of different application domains. Recently, the emergence of large language models (LLMs) has introduced new opportunities to enhance the design study process, providing capabilities such as creative problem-solving, data handling, and insightful analysis. However, despite their growing popularity, there remains a lack of systematic understanding of how LLMs can effectively assist researchers in visualization-specific design studies. In this paper, we conducted a multi-stage qualitative study to fill this gap, involving 30 design study researchers from diverse backgrounds and expertise levels. Through in-depth interviews and carefully-designed questionnaires, we investigated strategies for utilizing LLMs, the challenges encountered, and the practices used to overcome them. We further compiled and summarized the roles that LLMs can play across different stages of the design study process. Our findings highlight practical implications to inform visualization practitioners, and provide a framework for leveraging LLMs to enhance the design study process in visualization research.

cs.HC↗

arXiv2025

AutoPLC: Generating Vendor-Aware Structured Text for Programmable Logic Controllers

Among the programming languages for Programmable Logic Controllers (PLCs), Structured Text (ST) is widely adopted for industrial automation due to its expressiveness and flexibility. However, major vendors implement ST with proprietary extensions and hardware-specific libraries - Siemens' SCL and CODESYS' ST each differ in syntax and functionality. This fragmentation forces engineers to relearn implementation details across platforms, creating substantial productivity barriers. To address this challenge, we developed AutoPLC, a framework capable of automatically generating vendor-aware ST code directly from natural language requirements. Our solution begins by building two essential knowledge sources tailored to each vendor's specifications: a structured API library containing platform-exclusive functions, and an annotated case database that captures real-world implementation experience. Building on these foundations, we created a four-stage generation process that combines step-wise planning (enhanced with a lightweight natural language state machine support for control logic), contextual case retrieval using LLM-based reranking, API recommendation guided by industrial data, and dynamic validation through direct interaction with vendor IDEs. Implemented for Siemens TIA Portal and the CODESYS platform, AutoPLC achieves 90%+ compilation success on our 914-task benchmark (covering general-purpose and process control functions), outperforming all selected baselines, at an average cost of only $0.13 per task. Experienced PLC engineers positively assessed the practical utility of the generated code, including cases that failed compilation. We open-source our framework at https://github.com/cangkui/AutoPLC.

cs.SE↗

arXiv2025

Revealing Atomic-Scale Switching Pathways in van der Waals Ferroelectrics

Two-dimensional van der Waals (vdW) materials hold the potential for ultra-scaled ferroelectric (FE) devices due to their silicon compatibility and robust polarization down to atomic scale. However, the inherently weak vdW interactions enable facile sliding between layers, introducing complexities beyond those encountered in conventional ferroelectric materials and presenting significant challenges in uncovering intricate switching pathways. Here, we combine atomic-resolution imaging under in-situ electrical biasing conditions with first-principles calculations to unravel the atomic-scale switching mechanisms in SnSe, a vdW group-IV monochalcogenide. Our results uncover the coexistence of a consecutive 90 degrees switching pathway and a direct 180 degrees switching pathway from antiferroelectric (AFE) to FE order in this vdW system. Atomic-scale investigations and strain analysis reveal that the switching processes simultaneously induce interlayer sliding and compressive strain, while the lattice remains coherent despite the presence of multidomain structures. These findings elucidate vdW ferroelectric switching dynamics at atomic scale and lay the foundation for the rational design of 2D ferroelectric nanodevices.

cond-mat.mtrl-sci↗

arXiv2025

Selective Prompt Anchoring for Code Generation

Recent advances in large language models (LLMs) have transformed software development by automatically generating code from natural language. Yet challenges remain in generating fully correct code that aligns with user intent. Our study reveals that LLMs tend to pay less attention to user prompts as more code tokens are generated. We hypothesize that this attention dilution issue is an important reason for code generation errors. To mitigate this issue, we propose Selective Prompt Anchoring (SPA) to guide code LLMs to pay more attention to user intent when generating code. We evaluate SPA using six base LLMs across six benchmarks. Our results demonstrate that SPA enhances Pass@1 by up to 12.9%, consistently outperforming SOTA code generation methods in all settings. Our code is available at https://github.com/magic-YuanTian/Selective-Prompt-Anchoring.

cs.LG↗

arXiv2025

Algorithm Design and Comparative Test of Natural Gradient Gaussian Approximation Filter

Popular Bayes filters typically rely on linearization techniques such as Taylor series expansion and stochastic linear regression to use the structure of standard Kalman filter. These techniques may introduce large estimation errors in nonlinear and non-Gaussian systems. This paper overviews a recent breakthrough in filtering algorithm design called \textit{N}atural Gr\textit{a}dient Gaussia\textit{n} Appr\textit{o}ximation (NANO) filter and compare its performance over a large class of nonlinear filters. The NANO filter interprets Bayesian filtering as solutions to two distinct optimization problems, which allows to define optimal Gaussian approximation and derive its corresponding extremum conditions. The algorithm design still follows the two-step structure of Bayes filters. In the prediction step, NANO filter calculates the first two moments of the prior distribution, and this process is equivalent to a moment-matching filter. In the update step, natural gradient descent is employed to directly minimize the objective of the update step, thereby avoiding errors caused by model linearization. Comparative tests are conducted on four classic systems, including the damped linear oscillator, sequence forecasting, modified growth model, and robot localization, under Gaussian, Laplace, and Beta noise to evaluate the NANO filter's capability in handling nonlinearity. Additionally, we validate the NANO filter's robustness to data outliers using a satellite attitude estimation example. It is observed that the NANO filter outperforms popular Kalman filters family such as extended Kalman filter (EKF), unscented Kalman filter (UKF), iterated extended Kalman filter (IEKF) and posterior linearization filter (PLF), while having similar computational burden.

eess.SY↗

arXiv2025

Paths and Intersections: Exact Emulators for Planar Graphs

We study vertex sparsification for preserving distances in planar graphs. Given an edge-weighted planar graph with $k$ terminals, the goal is to construct an emulator, which is a smaller edge-weighted planar graph that contains the terminals and exactly preserves the pairwise distances between them. We construct exact planar emulators of size $O(f^2k^2)$ in the setting where terminals lie on $f$ faces in the planar embedding of the input graph. Our result generalizes and interpolates between the previous results of Chang and Ophelders and Goranci, Henzinger, and Peng which is an $O(k^2)$ bound in the setting where all terminals lie on a single face (i.e., $f=1$), and the result of Krauthgamer, Nguyen, and Zondiner, which is an $O(k^4)$ bound for the general case (i.e., $f=k$). Our construction follows a recent new way of analyzing graph structures, by viewing graphs as paths and their intersections, which we believe is of independent interest.

cs.DS↗

arXiv2025

On some notions of rank for matrices over tracts

Given a tract $F$ in the sense of Baker and Bowler and a matrix $A$ with entries in $F$, we define several notions of rank for $A$. In this way, we are able to unify and find conceptually satisfying proofs for various results about ranks of matrices that one finds scattered throughout the literature.

math.CO↗

arXiv2025

CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems

Recommender systems play a pivotal role in providing relevant content to users. With the rapid development of large language models (LLMs), researchers have begun utilizing LLMs to build more powerful recommender systems. However, existing approaches that focus on aligning LLMs with recommendation tasks do not fully leverage their sequential information processing capabilities, leading to suboptimal performance. In this paper, we propose a novel system called compressed vocabulary expansion (CoVE). In CoVE, each item is assigned a unique ID within the expanded vocabulary. Our framework effectively capitalizes on sequence understanding abilities of LLMs, significantly enhancing their performance on recommendation tasks. Additionally, we compress the embedding layer, making CoVE practical for large-scale industrial applications. The effectiveness and performance of CoVE are demonstrated through comprehensive experiments on multiple recommendation datasets and comparisons with prior works. Our code can be found at https://github.com/HaochenZhang717/CoVE-official-Repo.

cs.IR↗

arXiv2025

Multi-modal Traffic Scenario Generation for Autonomous Driving System Testing

Autonomous driving systems (ADS) require extensive testing and validation before deployment. However, it is tedious and time-consuming to construct traffic scenarios for ADS testing. In this paper, we propose TrafficComposer, a multi-modal traffic scenario construction approach for ADS testing. TrafficComposer takes as input a natural language (NL) description of a desired traffic scenario and a complementary traffic scene image. Then, it generates the corresponding traffic scenario in a simulator, such as CARLA and LGSVL. Specifically, TrafficComposer integrates high-level dynamic information about the traffic scenario from the NL description and intricate details about the surrounding vehicles, pedestrians, and the road network from the image. The information from the two modalities is complementary to each other and helps generate high-quality traffic scenarios for ADS testing. On a benchmark of 120 traffic scenarios, TrafficComposer achieves 97.0% accuracy, outperforming the best-performing baseline by 7.3%. Both direct testing and fuzz testing experiments on six ADSs prove the bug detection capabilities of the traffic scenarios generated by TrafficComposer. These scenarios can directly discover 37 bugs and help two fuzzing methods find 33%--124% more bugs serving as initial seeds.

cs.SE↗

arXiv2025

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference

Learning reward models from human preference datasets and subsequently optimizing language models via reinforcement learning has emerged as a fundamental paradigm for aligning LLMs with human preferences. The performance of the reward model plays a crucial role in the effectiveness of alignment. Previous reward models operate at a coarse-grained level, requiring the generation of a complete response to obtain a reward value. The sparse reward may present challenges for downstream reinforcement learning. While recent efforts have attempted to learn token-level reward models, the lack of explicit semantic information makes it difficult to model the credit of every individual token. In this paper, we propose assigning scores to every sentence, introducing an intermediate-grained reward model. By segmenting the complete response into sentences and applying differential operations to reward output at the start and end positions of each sentence, we can effectively model the rewards of sentences. Moreover, a novel attention mechanism is introduced to aggregate the scores of all sentences into a response-level score, which allows it to be trained using the Bradley-Terry model. On common benchmarks, our method outperforms the response-level reward model by 2.7% on RewardBench (for reward modeling evaluation) and surpasses all baselines on AlpacaEval (for alignment evaluation).

cs.CL↗

arXiv2025

Incorporating Uncertainty-Guided and Top-k Codebook Matching for Real-World Blind Image Super-Resolution

Recent advancements in codebook-based real image super-resolution (SR) have shown promising results in real-world applications. The core idea involves matching high-quality image features from a codebook based on low-resolution (LR) image features. However, existing methods face two major challenges: inaccurate feature matching with the codebook and poor texture detail reconstruction. To address these issues, we propose a novel Uncertainty-Guided and Top-k Codebook Matching SR (UGTSR) framework, which incorporates three key components: (1) an uncertainty learning mechanism that guides the model to focus on texture-rich regions, (2) a Top-k feature matching strategy that enhances feature matching accuracy by fusing multiple candidate features, and (3) an Align-Attention module that enhances the alignment of information between LR and HR features. Experimental results demonstrate significant improvements in texture realism and reconstruction fidelity compared to existing methods. We will release the code upon formal publication.

cs.CV↗