arXiv Science⌕ Search

arXiv subjects

Lingfang Li

Publications and source records attributed to Lingfang Li.

6 recordsLinked to original sources

When Self-Consistency Hurts: The Hidden Cost of Majority Voting in Small Language Models

We study prompted chain-of-thought (CoT) with self-consistency (SC), i.e. sampling N traces and plurality-voting, on small language models, a regime where per-query compute budgets are tight yet SC is commonly applied with a fixed sample budget inherited from large-model practice, regardless of instance difficulty. Across two model families (Qwen-2.5-0.5B, Gemma-3-1B) at FP16 and MLX-Q4 (4-bit quantisation in Apple's MLX framework), on GSM8K and MATH-500, we run 8 per-task and 4 task-pooled experiments at n=200-400 with 1,000x paired-bootstrap confidence intervals (CIs). Our first result is an upper bound: an oracle gate deciding per prompt whether to invoke SC reaches 109-174% of always-SC's accuracy at 23-30% of its compute in all 12/12 cases, so at N=5 always-SC is dominated everywhere on our grid. That margin is exactly the rate at which SC flips correct greedy answers to wrong, here 3-10.5% of prompts, which on Qwen-Q4 x MATH-500 exceeds the wrong-to-right rate and leaves always-SC 3 pt worse than always-greedy. Our second result is a deployable approximation: a gate over greedy-pass signals skips 45-99% of SC calls, with bootstrap CI half-widths under 5 pt, removing 36-79% of always-SC compute. Accuracy retention at those operating points has point estimates of 98.1-98.8%, which we report as point estimates because their CIs are wide (lower bounds 76-86%) over an SC accuracy of only 15-36%. The evidence covers N=5 on two <=1B families and two math benchmarks, which is the scope we claim. Code is available at https://github.com/heywanrong/SiftSC.

cs.AI↗

Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Models

Fine-tuning has emerged as a widely adopted approach for adapting LLMs to a variety of downstream tasks. However, how it reshapes their internal mechanisms remains poorly understood. To address this, we investigate how fine-tuning alters internal representations in LLMs, including attention patterns and layer-wise activations, and examine whether these changes are linked to task-relevant components identified by EAP (e.g., attention heads and logit-level activations) that drive task performance. We find that EAP-identified components are concentrated within specific layers, indicating a degree of functional localisation in how models internalise task-specific behavior. Notably, the distribution of these components across layers is largely uncorrelated with the layers undergoing the most substantial representational changes during fine-tuning. Furthermore, we observe that overlap in EAP-identified components across tasks does not translate into cross-task performance transfer if the tasks are different in nature (e.g. classification vs. generative tasks). More specifically, fine-tuning on one task can lead to a degradation of performance on another when the two tasks exhibit a high degree of overlap in their EAP-identified components.

cs.AI↗

Non-Hermitian topology driven by an identity term: An exactly solvable paradigm

An identity term in the Hamiltonian is conventionally regarded as spectrally inert-it shifts energies but does not alter eigenstate topology. We show that under non-Hermitian skin pumping, this paradigm fails: a momentum-dependent identity term actively deforms the generalized Brillouin zone, thereby challenging established topological criteria that rely on fixed complex contours. Here, by introducing spin-orbit coupling into a Hatano-Nelson chain, we present an exact analytical solution for the entire non-Hermitian eigensystem under open boundary conditions. Our solution reveals how inter-cell spin-orbit coupling, synergizing with this non-trivial identity term, induces topological edge states and robust zero modes in the complete absence of chiral symmetry. This work establishes an exactly solvable paradigm for non-Hermitian topology beyond symmetry protection, and provides a rigorous benchmark for testing topological invariants in systems with momentum-dependent identity terms.

quant-ph↗

Analyzing the Effect of Noise in LLM Fine-tuning

Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may contain various forms of noise arising from annotation errors, preprocessing artifacts, or automated data collection. While prior work has focused on designing robust learning algorithms to mitigate performance degradation under noisy conditions, comparatively little is known about how different types of noise affect the internal learning dynamics of LLMs during fine-tuning. In this work, we systematically study the impact of noise on model behavior across three pretrained model families (GPT-2, Qwen2 and Llama-2) and three diverse NLP tasks. We introduce controlled perturbations corresponding to three common real-world noise types: label noise, grammatical noise, and typographical noise. Beyond task-level performance, we analyze layer-wise representation changes and attention patterns to understand how noise propagates through the network. Our results show that corrupting labels (i.e. label noise) consistently causes the largest performance degradation, whereas grammatical noise and typographical noise can occasionally yield mild regularization benefits. We further find that noise effects are localized primarily to task-specific layers, while attention structures remain comparatively stable.

cs.LG↗

Empowering Locally Deployable Medical Agent via State Enhanced Logical Skills for FHIR-based Clinical Tasks

While Large Language Models demonstrate immense potential as proactive Medical Agents, their real-world deployment is severely bottlenecked by data scarcity under privacy constraints. To overcome this, we propose State-Enhanced Logical-Skill Memory (SELSM), a training-free framework that distills simulated clinical trajectories into entity-agnostic operational rules within an abstract skill space. During inference, a Query-Anchored Two-Stage Retrieval mechanism dynamically fetches these entity-agnostic logical priors to guide the agent's step-by-step reasoning, effectively resolving the state polysemy problem. Evaluated on MedAgentBench -- the only authoritative high-fidelity virtual EHR sandbox benchmarked with real clinical data -- SELSM substantially elevates the zero-shot capabilities of locally deployable foundation models (30B--32B parameters). Notably, on the Qwen3-30B-A3B backbone, our framework completely eliminates task chain breakdowns to achieve a 100\% completion rate, boosting the overall success rate by an absolute 22.67\% and significantly outperforming existing memory-augmented baselines. This study demonstrates that equipping models with a dynamically updatable, state-enhanced cognitive scaffold is a privacy-preserving and computationally efficient pathway for local adaptation of AI agents to clinical information systems. While currently validated on FHIR-based EHR interactions as an initial step, the entity-agnostic design of SELSM provides a principled foundation toward broader clinical deployment.

cs.AI↗

Exact Solutions Disentangle Higher-Order Topology in 2D Non-Hermitian Lattices

We report the exact closed-form solutions for higher-order topological states as well as explicit energy-spectrum relationships in two-dimensional (2D) non-Hermitian multi-orbital lattices with generalized boundary conditions. These analytical solutions unequivocally confirm that topological edge states in a 2D non-Hermitian system which feature point-gap topology must undergo the non-Hermitian skin effect along the edge. Under double open boundary conditions, the occurrence of the non-Hermitian skin effect for either topological edge states or bulk states can be accurately predicted by our proposed winding numbers. We unveil that the zero-energy topological corner state only manifests itself on a corner where two nearby gapped edge states intersect, and thus can either disappear completely or strengthen drastically due to the non-Hermitian skin effect of gapped topological edge states. Our analytical results offer direct insight into the non-Bloch band topology in two or higher dimensions and trigger experimental investigations into related phenomena such as quadrupole topological insulators and topological lasing.

physics.optics↗