arXiv Science⌕ Search

arXiv subjects

Yusu Zhang

Publications and source records attributed to Yusu Zhang.

4 recordsLinked to original sources

Disagree to Prune: Reallocating Visual Tokens Where a Small VLM Dissents

Visual token pruning speeds up large vision-language models (VLMs) by keeping only a part of their visual tokens. Most existing methods improve the ranking of the tokens and then answer every question once with the same budget. With strong rankings, pruning hurts only a few questions, and signals computed on the kept tokens see little of what it removed. We find that, on most of these questions, a small VLM given the full image disagrees with the pruned answer, since it still sees the missing evidence. We propose PRUDENT, which first answers with about half the budget and spends the saved tokens on re-answering the disagreeing questions at twice the budget. We show when such re-answering pays and why the small VLM should read the full image. On 18 benchmarks, PRUDENT outperforms eight existing pruning methods at the same average budget of visual tokens. Its lead widens as the budget tightens, and holds across five InternVL sizes. PRUDENT also has the highest average in two more model families. The disagreement of the small VLM flags most pruning failures of all eight methods, and re-answering on it also improves the two strongest with their own rankings. Other checkers give similar gains, and the extra tokens go where the first pass hurts the answer. These results suggest that identifying which pruned answers are trustworthy provides an additional lever for visual token pruning beyond the ranking of tokens. Code and data are available at https://github.com/LiZaiyuan0619/PRUDENT.

cs.CV↗

Prior-guided adaptive local refinement of foundation interatomic potentials

Foundation interatomic potentials enable simulations across diverse materials, but task-specific inaccuracies can require additional first-principles data. Obtaining these data is challenging when the relevant local environments arise in large simulation cells. Here we introduce the Prior-guided Adaptive Refinement Scheme (PARS), which identifies weakly supported environments during molecular dynamics by comparing their frozen representations with prototypes of pretraining environments, without requiring initial task labels or model ensembles. Selected environments are reconstructed as compact periodic cells for first-principles labelling and iterative refinement. Applications to phase stability, surface migration and vacancy clustering show improvements in target properties, including an 81% reduction in the mean absolute error of Ti vacancy-cluster energies per vacancy relative to the pretrained model. For Si/Ge, a foundation potential refined using 800 labelled cells containing 10-128 atoms is subsequently distilled into a compact potential without additional first-principles training labels. In simulations containing approximately 362,880 atoms, the distilled model predicts an interfacial thermal resistance consistent with that of a potential trained directly on density-functional theory data. These results demonstrate a route for adapting foundation potentials to target applications through local reference calculations.

cond-mat.mtrl-sci↗

PACT: Certifying What Is Written in Parallel Decoding of Diffusion Language Models

Parallel decoding speeds up masked diffusion language models by writing several tokens per forward pass. Published commit rules choose these tokens by testing the predicted distributions. Under sampling the written tokens are drawn, and these tests do not see them. We find that answer quality follows the reliability of the tokens actually written. An answer's log probability splits into this reliability and the dependence within each step. We propose PACT, a training-free commit rule that draws the tokens first, certifies the set it will write and spaces the positions of each step apart. This certificate holds at every temperature, and we show that no rule with the same guarantee commits more positions. Experiments use four models of the LLaDA and Dream families and HumanEval, MBPP, GSM8K, MATH500 and IFEval benchmarks. PACT reaches the quality of published commit rules with fewer forward passes. Spacing removes positions from each step, yet it usually lowers the total cost because later steps then write more tokens. At the same speed, published decoders also write a word twice in a row far more often than PACT. These results suggest that commit rules should certify the tokens they write. Code is available at https://github.com/LiZaiyuan0619/PACT.

cs.CL↗

Weighted balanced truncation method for approximating kernel functions by exponentials

Kernel approximation with exponentials is useful in many problems with convolution quadrature and particle interactions such as integral-differential equations, molecular dynamics and machine learning. This paper proposes a weighted balanced truncation to construct an optimal model reduction method for compressing the number of exponentials in the sum-of-exponentials approximation of kernel functions. This method shows great promise in approximating long-range kernels, achieving over 4 digits of accuracy improvement for the Ewald-splitting and inverse power kernels in comparison with the classical balanced truncation. Numerical results demonstrate its excellent performance and attractive features for practical applications.

physics.comp-ph↗