arXiv subjects

Shuo Li

Publications and source records attributed to Shuo Li.

At least 127 records · Page 7Linked to original sources

arXiv2025

WST: Weak-to-Strong Knowledge Transfer via Reinforcement Learning

Effective prompt engineering remains a challenging task for many applications. We introduce Weak-to-Strong Transfer (WST), an automatic prompt engineering framework where a small "Teacher" model generates instructions that enhance the performance of a much larger "Student" model. Unlike prior work, WST requires only a weak teacher, making it efficient and broadly applicable in settings where large models are closed-source or difficult to fine-tune. Using reinforcement learning, the Teacher Model's instructions are iteratively improved based on the Student Model's outcomes, yielding substantial gains across reasoning (MATH-500, GSM8K) and alignment (HH-RLHF) benchmarks - 98% on MATH-500 and 134% on HH-RLHF - and surpassing baselines such as GPT-4o-mini and Llama-70B. These results demonstrate that small models can reliably scaffold larger ones, unlocking latent capabilities while avoiding misleading prompts that stronger teachers may introduce, establishing WST as a scalable solution for efficient and safe LLM prompt refinement.

cs.LG↗

arXiv2025

Information Bottleneck-based Causal Attention for Multi-label Medical Image Recognition

Multi-label classification (MLC) of medical images aims to identify multiple diseases and holds significant clinical potential. A critical step is to learn class-specific features for accurate diagnosis and improved interpretability effectively. However, current works focus primarily on causal attention to learn class-specific features, yet they struggle to interpret the true cause due to the inadvertent attention to class-irrelevant features. To address this challenge, we propose a new structural causal model (SCM) that treats class-specific attention as a mixture of causal, spurious, and noisy factors, and a novel Information Bottleneck-based Causal Attention (IBCA) that is capable of learning the discriminative class-specific attention for MLC of medical images. Specifically, we propose learning Gaussian mixture multi-label spatial attention to filter out class-irrelevant information and capture each class-specific attention pattern. Then a contrastive enhancement-based causal intervention is proposed to gradually mitigate the spurious attention and reduce noise information by aligning multi-head attention with the Gaussian mixture multi-label spatial. Quantitative and ablation results on Endo and MuReD show that IBCA outperforms all methods. Compared to the second-best results for each metric, IBCA achieves improvements of 6.35\% in CR, 7.72\% in OR, and 5.02\% in mAP for MuReD, 1.47\% in CR, and 1.65\% in CF1, and 1.42\% in mAP for Endo.

cs.CV↗

arXiv2025

Universal Magnetocaloric Effect near Quantum Critical Point of Magnon Bose-Einstein Condensation

Bose-Einstein condensation (BEC), a macroscopic quantum phenomenon arising from phase coherence and bosonic statistics, has been realized in quantum magnets. Here, we report the observation of a universal magnetocaloric effect (MCE) near a BEC quantum critical point (QCP) in copper sulfate crystal ($CuSO_4 \cdot 5H_2O$). By conducting magnetocaloric and nuclear magnetic resonance measurements, we uncover a field-driven BEC QCP, evidenced by the universal scaling law $T_c \propto (B_c - B)^{2/3}$ and the perfect data collapse of the magnetic Grüneisen ratio. Thermal excitation triggers a dimensional crossover to a 1D quantum-critical regime, where the MCE scaling strictly matches the universality class of 1D Fermi gases. Notably, the quantum-critical MCE enables cooling down to 12.8 mK without helium-3, with very fast thermal relaxation rate that is critical for high cooling power. This work demonstrates the universal MCE in magnon BEC systems, using a common copper sulfate compound as a paradigmatic example, and paves the way for next-generation sub-Kelvin cooling.

cond-mat.str-el↗

arXiv2025

Vector Contrastive Learning For Pixel-Wise Pretraining In Medical Vision

Contrastive learning (CL) has become a cornerstone of self-supervised pretraining (SSP) in foundation models, however, extending CL to pixel-wise representation, crucial for medical vision, remains an open problem. Standard CL formulates SSP as a binary optimization problem (binary CL) where the excessive pursuit of feature dispersion leads to an over-dispersion problem, breaking pixel-wise feature correlation thus disrupting the intra-class distribution. Our vector CL reformulates CL as a vector regression problem, enabling dispersion quantification in pixel-wise pretraining via modeling feature distances in regressing displacement vectors. To implement this novel paradigm, we propose the COntrast in VEctor Regression (COVER) framework. COVER establishes an extendable vector-based self-learning, enforces a consistent optimization flow from vector regression to distance modeling, and leverages a vector pyramid architecture for granularity adaptation, thus preserving pixel-wise feature correlations in SSP. Extensive experiments across 8 tasks, spanning 2 dimensions and 4 modalities, show that COVER significantly improves pixel-wise SSP, advancing generalizable medical visual foundation models.

cs.CV↗

arXiv2025

An LLM-Empowered Adaptive Evolutionary Algorithm For Multi-Component Deep Learning Systems

Multi-objective evolutionary algorithms (MOEAs) are widely used for searching optimal solutions in complex multi-component applications. Traditional MOEAs for multi-component deep learning (MCDL) systems face challenges in enhancing the search efficiency while maintaining the diversity. To combat these, this paper proposes $μ$MOEA, the first LLM-empowered adaptive evolutionary search algorithm to detect safety violations in MCDL systems. Inspired by the context-understanding ability of Large Language Models (LLMs), $μ$MOEA promotes the LLM to comprehend the optimization problem and generate an initial population tailed to evolutionary objectives. Subsequently, it employs adaptive selection and variation to iteratively produce offspring, balancing the evolutionary efficiency and diversity. During the evolutionary process, to navigate away from the local optima, $μ$MOEA integrates the evolutionary experience back into the LLM. This utilization harnesses the LLM's quantitative reasoning prowess to generate differential seeds, breaking away from current optimal solutions. We evaluate $μ$MOEA in finding safety violations of MCDL systems, and compare its performance with state-of-the-art MOEA methods. Experimental results show that $μ$MOEA can significantly improve the efficiency and diversity of the evolutionary search.

cs.NE↗

arXiv2025

Two Plaquette-Singlet Phases and Emergent SO(5) Deconfined Quantum Criticality in SrCu2(BO3)2

The deconfined quantum critical point (DQCP) has become a central open concept in the physics of quantum matter, and its proposed presence in the Shastry-Sutherland model was followed by the experimental observation of at least a minimal DQC scenario induced by an applied magnetic field in SrCu$_2$(BO$_3$)$_2$. However, the nature of the plaquette-singlet phase in SrCu$_2$(BO$_3$)$_2$ remains unresolved, and with it the identification of the DQCP symmetry from among several theoretical scenarios. Here we perform detailed high-pressure $^{11}$B NMR studies to reveal the presence of both the full-plaquette (FP) and empty-plaquette (EP) phases in SrCu$_2$(BO$_3$)$_2$, phase-separated at a first-order, pressure-driven transition with a volume-fraction effect. The field-driven transition from the EP to the antiferromagnetic (AFM) phase complements our previous observations of the FP--AFM transition, with both showing deconfined quantum criticality, while the scaling of the spin-lattice relaxation rate near the EP--AFM transition, $1/T_1 \propto T^{0.6}$, suggests a DQCP governed by a different universality class. We discuss possible extensions to the Shastry-Sutherland model that account for these pressure and field effects. The expanded phase space we discover mandates an SO(5) DQCP symmetry, and hence our results take an important step towards a complete understanding of deconfined quantum criticality in SrCu$_2$(BO$_3$)$_2$.

cond-mat.str-el↗

arXiv2025

Ultrafast magneto-lattice dynamics in two-dimensional CrSBr driven by terahertz excitation

Terahertz (THz) lasers provide a new research perspective for spin electronics applications due to their sub-picosecond time resolution and non-thermal ultrafast demagnetization, but the interaction between spin, charge and lattice dynamics remains unclear. This study investigates photoinduced ultrafast demagnetization in monolayer CrSBr, a two-dimensional material with strong spin-orbit and spin-lattice coupling, and resolves its demagnetization process. Two key stages are identified: the first, occurring within 20 fs, is characterized by rapid electron-driven demagnetization, where charge transfer and THz laser are strongly coupled. In the second stage, light-induced lattice vibrations coupled to spin dynamics lead to significant spin changes, with electron-phonon coupling playing a key role. Importantly, the role of various phonon vibration modes in the electron relaxation process was clearly determined, pointing out that the electronic relaxation of the B3g1 phonon vibration mode occurs within 83 fs, which is less than the commonly believed 100 fs. Moreover, the influence of this coherent phonon on the demagnetization change is as high as 215 %. These insights into multiscale magneto-structural coupling advance the understanding of nonequilibrium spin dynamics and provide guidelines for the design of light-controlled quantum devices, particularly in layered heterostructures for spintronics and quantum information technologies.

cond-mat.mtrl-sci↗

arXiv2025

Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric

Data diversity is crucial for the instruction tuning of large language models. Existing studies have explored various diversity-aware data selection methods to construct high-quality datasets and enhance model performance. However, the fundamental problem of precisely defining and measuring data diversity remains underexplored, limiting clear guidance for data engineering. To address this, we systematically analyze 11 existing diversity measurement methods by evaluating their correlation with model performance through extensive fine-tuning experiments. Our results indicate that a reliable diversity measure should properly account for both inter-sample differences and the information density in the sample space. Building on this, we propose NovelSum, a new diversity metric based on sample-level "novelty." Experiments on both simulated and real-world data show that NovelSum accurately captures diversity variations and achieves a 0.97 correlation with instruction-tuned model performance, highlighting its value in guiding data engineering practices. With NovelSum as an optimization objective, we further develop a greedy, diversity-oriented data selection strategy that outperforms existing approaches, validating both the effectiveness and practical significance of our metric. The code is available at https://github.com/UmeanNever/NovelSum.

cs.CL↗

arXiv2025

DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms

Dongba pictographic is the only pictographic script still in use in the world. Its pictorial ideographic features carry rich cultural and contextual information. However, due to the lack of relevant datasets, research on semantic understanding of Dongba hieroglyphs has progressed slowly. To this end, we constructed \textbf{DongbaMIE} - the first dataset focusing on multimodal information extraction of Dongba pictographs. The dataset consists of images of Dongba hieroglyphic characters and their corresponding semantic annotations in Chinese. It contains 23,530 sentence-level and 2,539 paragraph-level high-quality text-image pairs. The annotations cover four semantic dimensions: object, action, relation and attribute. Systematic evaluation of mainstream multimodal large language models shows that the models are difficult to perform information extraction of Dongba hieroglyphs efficiently under zero-shot and few-shot learning. Although supervised fine-tuning can improve the performance, accurate extraction of complex semantics is still a great challenge at present.

cs.CV↗

arXiv2025

Effective Reinforcement Learning for Reasoning in Language Models

Reinforcement learning (RL) has emerged as a promising strategy for improving the reasoning capabilities of language models (LMs) in domains such as mathematics and coding. However, most modern RL algorithms were designed to target robotics applications, which differ significantly from LM reasoning. We analyze RL algorithm design decisions for LM reasoning, for both accuracy and computational efficiency, focusing on relatively small models due to computational constraints. Our findings are: (i) on-policy RL significantly outperforms supervised fine-tuning (SFT), (ii) PPO-based off-policy updates increase accuracy instead of reduce variance, and (iii) removing KL divergence can lead to more concise generations and higher accuracy. Furthermore, we find that a key bottleneck to computational efficiency is that the optimal batch sizes for inference and backpropagation are different. We propose a novel algorithm, DASH, that performs preemptive sampling (i.e., sample a large batch and accumulate gradient updates in small increments), and gradient filtering (i.e., drop samples with small advantage estimates). We show that DASH reduces training time by 83% compared to a standard implementation of GRPO without sacrificing accuracy. Our findings provide valuable insights on designing effective RL algorithms for LM reasoning.

cs.AI↗

arXiv2025

Quantifying the advantage of quantum correlation microscopy using arrays of single-photon detectors

Quantum correlation microscopy is an emerging technique for improving optical resolution. By taking advantage of the quantum statistics from single-photon fluorophores, more information about the emitters (including number and location) is obtained compared with classical microscopy. Although it is known that the resolution can be improved by increasing detector numbers, as well as using quantum correlation, the quantitative relationship between these two approaches is not immediately clear. Here we explore widefield quantum correlation microscopy using arrays of single-photon detectors. We explicitly compare the use of $N$ detectors used in photon counting mode vs $N/2$ detectors used to measure quantum correlations. i.e., where there are $N/2$ Hanbury Brown and Twiss systems, using the same $N$ detectors, on randomly generated two-emitter systems. We find regimes where $N/2$ Hanbury Brown and Twiss detectors provide improved localisation compared to $N$ photon counting detectors, as a function of emitter position and number of photons sampled.

physics.optics↗

arXiv2025

Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput

In this paper, we introduce Flash-VL 2B, a novel approach to optimizing Vision-Language Models (VLMs) for real-time applications, targeting ultra-low latency and high throughput without sacrificing accuracy. Leveraging advanced architectural enhancements and efficient computational strategies, Flash-VL 2B is designed to maximize throughput by reducing processing time while maintaining competitive performance across multiple vision-language benchmarks. Our approach includes tailored architectural choices, token compression mechanisms, data curation, training schemes, and a novel image processing technique called implicit semantic stitching that effectively balances computational load and model performance. Through extensive evaluations on 11 standard VLM benchmarks, we demonstrate that Flash-VL 2B achieves state-of-the-art results in both speed and accuracy, making it a promising solution for deployment in resource-constrained environments and large-scale real-time applications.

cs.CV↗

arXiv2025

The reflection complexity of sequences over finite alphabets

In combinatorics on words, the well-studied factor complexity function $ρ_{\infw{x}}$ of a sequence $\infw{x}$ over a finite alphabet counts, for every nonnegative integer $n$, the number of distinct length-$n$ factors of $\infw{x}$. In this paper, we introduce the \emph{reflection complexity} function $r_{\infw{x}}$ to enumerate the factors occurring in a sequence $\infw{x}$, up to reversing the order of symbols in a word. We prove a number of results about the growth properties of $r_{\infw{x}}$ and its relationship with other complexity functions. We also prove a Morse--Hedlund-type result characterizing eventually periodic sequences in terms of their reflection complexity, and we deduce a characterization of Sturmian sequences. We investigate the reflection complexity of quasi-Sturmian, episturmian, $(s+1)$-dimensional billiard, complementation-symmetric Rote, and rich sequences. Furthermore, we prove that if $\infw{x}$ is $k$-automatic, then $r_{\infw{x}}$ is computably $k$-regular, and we use the software \texttt{Walnut} to evaluate the reflection complexity of some automatic sequences, such as the Thue--Morse sequence. We note that there are still many unanswered questions about this reflection measure.

math.CO↗

arXiv2025

Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation

Large Language Models (LLMs) often struggle to process and generate coherent context when the number of input tokens exceeds the pre-trained length. Recent advancements in long-context extension have significantly expanded the context window of LLMs but require expensive overhead to train the large-scale models with longer context. In this work, we propose Dimension-Wise Positional Embeddings Manipulation (DPE), a training-free framework to extrapolate the context window of LLMs by diving into RoPE's different hidden dimensions. Instead of manipulating all dimensions equally, DPE detects the effective length for every dimension and finds the key dimensions for context extension. We reuse the original position indices with their embeddings from the pre-trained model and manipulate the key dimensions' position indices to their most effective lengths. In this way, DPE adjusts the pre-trained models with minimal modifications while ensuring that each dimension reaches its optimal state for extrapolation. DPE significantly surpasses well-known baselines such as YaRN and Self-Extend. DPE enables Llama3-8k 8B to support context windows of 128k tokens without continual training and integrates seamlessly with Flash Attention 2. In addition to its impressive extrapolation capability, DPE also dramatically improves the models' performance within training length, such as Llama3.1 70B, by over 18 points on popular long-context benchmarks RULER. When compared with commercial models, Llama 3.1 70B with DPE even achieves better performance than GPT-4-128K.

cs.CL↗

arXiv2025

Logits DeConfusion with CLIP for Few-Shot Learning

With its powerful visual-language alignment capability, CLIP performs well in zero-shot and few-shot learning tasks. However, we found in experiments that CLIP's logits suffer from serious inter-class confusion problems in downstream tasks, and the ambiguity between categories seriously affects the accuracy. To address this challenge, we propose a novel method called Logits DeConfusion, which effectively learns and eliminates inter-class confusion in logits by combining our Multi-level Adapter Fusion (MAF) module with our Inter-Class Deconfusion (ICD) module. Our MAF extracts features from different levels and fuses them uniformly to enhance feature representation. Our ICD learnably eliminates inter-class confusion in logits with a residual structure. Experimental results show that our method can significantly improve the classification performance and alleviate the inter-class confusion problem. The code is available at https://github.com/LiShuo1001/LDC.

cs.CV↗

arXiv2025

NMR study of supersolid phases in the triangular-lattice antiferromagnet Na2BaCo(PO4)2

We report ultra-low-temperature $^{23}$Na NMR measurements on the Ising triangular lattice antiferromagnet Na$_2$BaCo(PO$_4$)$_2$, which precisely resolve the phase diagram under magnetic field applied along the crystalline $c$ axis. With increasing field, the NMR spectra resolve three ordered phases with distinct spin configurations: the Y, up-up-down (UUD), and V phases. The spin-lattice relaxation rate $1/T_1$ data demonstrate gapless excitations in the Y and V phases, strongly supporting their supersolid nature. However, the phase transitions from the UUD phase to the two supersolid phases exhibit dramatically different behaviors upon cooling. Prior to entering the Y phase, $1/T_1$ identifies a gapless regime within the UUD phase, suggesting a Berezinskii-Kosterlitz-Thouless phase above a second-order phase transition. In contrast, the coexistence of the UUD and V phases observed in our experiments provides direct evidence of a first-order phase transition between these phases.

cond-mat.str-el↗

arXiv2025

Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction

The use of AI in legal analysis and prediction (LegalAI) has gained widespread attention, with past research focusing on retrieval-based methods and fine-tuning large models. However, these approaches often require large datasets and underutilize the capabilities of modern large language models (LLMs). In this paper, inspired by the debate phase of real courtroom trials, we propose a novel legal judgment prediction model based on the Debate-Feedback architecture, which integrates LLM multi-agent debate and reliability evaluation models. Unlike traditional methods, our model achieves significant improvements in efficiency by minimizing the need for large historical datasets, thus offering a lightweight yet robust solution. Comparative experiments show that it outperforms several general-purpose and domain-specific legal models, offering a dynamic reasoning process and a promising direction for future LegalAI research.

cs.MA↗

arXiv2025

A Recent Supermassive Black Hole Binary in the Galactic Center Unveiled by the Hypervelocity Stars

When a binary of early-type stars from the young stellar populations in the Galactic center (GC) region is scattered to the vicinity of the supermassive black hole (SMBH) Sgr~$\rm{A}^{*}$, one of the components would be tidally ejected as an early-type hypervelocity star (HVS) and the counterpart would be captured on a tight orbit around Sgr~$\rm{A}^{*}$. Dozens of B-type HVSs moving faster than the Galactic escape speed have been discovered in the Galactic halo and are produced most likely by the SMBH Sgr~$\rm{A}^{*}$. However, the velocity distribution and in particular the deficit of the HVSs above $700\, \rm{km\, s^{-1}}$ is seriously inconsistent with the expectations of the present models. Here we show that the high-velocity deficit is due to the deficiency in close interactions of stars with the SMBH Sgr~$\rm{A}^{*}$, because an orbiting intermediate-mass black hole (IMBH) of about 15,000 Solar mass kicked away slowly approaching stars 50--250 million years ago. The SMBH-IMBH binary formed probably after the merger of the Milky Way with the Gaia-Sausage-Enceladus dwarf galaxy, and coalesced about 10 million years ago, leading to a gravitational recoil of Sgr~$\rm{A}^{*}$ at a velocity of 0.3--0.5$\, \rm{km\, s^{-1}}$ and to a change of the HVS ejection scenarios. The SMBH-IMBH binary scenario predicts the formation of the S-star cluster at the GC with the distribution of the orbital size and stellar ages that are well consistent with the observations.

astro-ph.HE↗