arXiv subjects

Yong Liu

Publications and source records attributed to Yong Liu.

At least 343 records · Page 19Linked to original sources

arXiv2025

EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflow

Medical Large Language Models (MLLMs) play a crucial role in ophthalmic diagnosis, holding significant potential to address vision-threatening diseases. However, their accuracy is constrained by hallucinations stemming from limited ophthalmic knowledge, insufficient visual localization and reasoning capabilities, and a scarcity of multimodal ophthalmic data, which collectively impede precise lesion detection and disease diagnosis. Furthermore, existing medical benchmarks fail to effectively evaluate various types of hallucinations or provide actionable solutions to mitigate them. To address the above challenges, we introduce EH-Benchmark, a novel ophthalmology benchmark designed to evaluate hallucinations in MLLMs. We categorize MLLMs' hallucinations based on specific tasks and error types into two primary classes: Visual Understanding and Logical Composition, each comprising multiple subclasses. Given that MLLMs predominantly rely on language-based reasoning rather than visual processing, we propose an agent-centric, three-phase framework, including the Knowledge-Level Retrieval stage, the Task-Level Case Studies stage, and the Result-Level Validation stage. Experimental results show that our multi-agent framework significantly mitigates both types of hallucinations, enhancing accuracy, interpretability, and reliability. Our project is available at https://github.com/ppxy1/EH-Benchmark.

cs.CL↗

arXiv2025

MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training

Foundation models have recently gained tremendous popularity in medical image analysis. State-of-the-art methods leverage either paired image-text data via vision-language pre-training or unpaired image data via self-supervised pre-training to learn foundation models with generalizable image features to boost downstream task performance. However, learning foundation models exclusively on either paired or unpaired image data limits their ability to learn richer and more comprehensive image features. In this paper, we investigate a novel task termed semi-supervised vision-language pre-training, aiming to fully harness the potential of both paired and unpaired image data for foundation model learning. To this end, we propose MaskedCLIP, a synergistic masked image modeling and contrastive language-image pre-training framework for semi-supervised vision-language pre-training. The key challenge in combining paired and unpaired image data for learning a foundation model lies in the incompatible feature spaces derived from these two types of data. To address this issue, we propose to connect the masked feature space with the CLIP feature space with a bridge transformer. In this way, the more semantic specific CLIP features can benefit from the more general masked features for semantic feature extraction. We further propose a masked knowledge distillation loss to distill semantic knowledge of original image features in CLIP feature space back to the predicted masked image features in masked feature space. With this mutually interactive design, our framework effectively leverages both paired and unpaired image data to learn more generalizable image features for downstream tasks. Extensive experiments on retinal image analysis demonstrate the effectiveness and data efficiency of our method.

cs.CV↗

arXiv2025

PromptAL: Sample-Aware Dynamic Soft Prompts for Few-Shot Active Learning

Active learning (AL) aims to optimize model training and reduce annotation costs by selecting the most informative samples for labeling. Typically, AL methods rely on the empirical distribution of labeled data to define the decision boundary and perform uncertainty or diversity estimation, subsequently identifying potential high-quality samples. In few-shot scenarios, the empirical distribution often diverges significantly from the target distribution, causing the decision boundary to shift away from its optimal position. However, existing methods overlook the role of unlabeled samples in enhancing the empirical distribution to better align with the target distribution, resulting in a suboptimal decision boundary and the selection of samples that inadequately represent the target distribution. To address this, we propose a hybrid AL framework, termed \textbf{PromptAL} (Sample-Aware Dynamic Soft \textbf{Prompts} for Few-Shot \textbf{A}ctive \textbf{L}earning). This framework accounts for the contribution of each unlabeled data point in aligning the current empirical distribution with the target distribution, thereby optimizing the decision boundary. Specifically, PromptAL first leverages unlabeled data to construct sample-aware dynamic soft prompts that adjust the model's predictive distribution and decision boundary. Subsequently, based on the adjusted decision boundary, it integrates uncertainty estimation with both global and local diversity to select high-quality samples that more accurately represent the target distribution. Experimental results on six in-domain and three out-of-domain datasets show that PromptAL achieves superior performance over nine baselines. Our codebase is openly accessible.

cs.CL↗

arXiv2025

SeWS/bilayer-SiC heterojunction: An S-scheme photocatalyst with high visible-light absorption, excellent carrier mobility and adjustable band gap

Vertically stacked heterojunctions have garnered significant attention for their tunable electronic structures and photocatalytic performance, making them promising candidates for next-generation nanodevices. Using first-principles calculations, we systematically investigate the electronic structure, optical characteristics, and charge transfer of WSSe/SiC heterojunctions. Our results reveal that SeWS/monolayer-SiC, SeWS/bilayer-SiC, and SWSe/monolayer-SiC exhibit type-II band alignment, whereas SWSe/bilayer-SiC displays type-I alignment. Notably, SeWS/bilayer-SiC possesses a direct bandgap, in contrast to the indirect bandgaps of the other three configurations. Remarkably, the SeWS/bilayer-SiC heterojunction demonstrates a high absorption coefficient ($10^{5}~\mathrm{cm}^{-1}$) in the visible range and exhibits exceptional anisotropy in carrier transport, with an outstanding hole mobility of $9.58 \times 10^{3}~\mathrm{cm}^{2}\,\mathrm{V}^{-1}\,\mathrm{s}^{-1}$ along the Y-direction. Furthermore, combining thermodynamic stability with an S-scheme charge transfer mechanism, this system exhibits superior redox capability for photocatalytic water splitting, achieving a high hydrogen evolution efficiency of 22.15%, which surpasses the commercial viability threshold (10\%). Furthermore, we demonstrate effective band gap modulation via external electric fields and biaxial strains, with optical absorption coefficients exhibiting strong strain dependence. This work provides fundamental insights into the design of WSSe/SiC heterojunctions for high-efficiency photocatalytic and tunable photodetector applications.

cond-mat.mtrl-sci↗

arXiv2025

Strain-induced nonrelativistic altermagnetic spin splitting effect

Recent studies reveal that $\mathcal{T}$-odd spin currents generated via the nonrelativistic altermagnetic spin splitting effect (ASSE) exhibit significant potential for spintronics applications, with both computational and experimental validations. Addressing the scarcity of conductive altermagnets, we propose strain engineering as a reliable method for inducing altermagnetism. Focusing on rutile-structured $\mathrm{OsO}_2$, first-principles calculations show that minor equibiaxial tensile strain ($\mathcal{E}_{\mathrm{ts}}$=3\%) induces nonmagnetic-to-altermagnetic transitions, achieving an ASSE-driven spin-charge conversion ratio ($θ_{\text{AS}}$) of $\sim$7\% -- far surpassing conventional spin Hall angles ($θ_{\text{IS}}$). Calculations reveal that substantial $θ_{\text{AS}}$ persists even in the absence of spin-orbit coupling, with its magnitude positively correlating to nonrelativistic spin splitting magnitude, which further confirms the strain-induced ASSE's nonrelativistic origin. Further investigation reveals that $\mathrm{RuO}_2$ exhibits analogous phenomena, which may resolve recent controversies regarding its magnetic properties. Our research opens new simple pathways for developing next-generation altermagnetic spintronic devices.

cond-mat.mtrl-sci↗

arXiv2025

Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger

Large language models (LLMs) have shown remarkable emergent capabilities, transforming the execution of functional tasks by leveraging external tools for complex problems that require specialized processing or up-to-date data. While existing research expands LLMs access to diverse tools (e.g., program interpreters, search engines, calculators), the necessity of using these tools is often overlooked, leading to indiscriminate tool invocation. This naive approach raises two key issues: increased latency due to unnecessary tool calls, and potential errors resulting from faulty interactions with external tools. In this paper, we introduce meta-cognition as a proxy for LLMs self-assessment of their capabilities, reflecting the model's awareness of its own limitations. Based on this, we propose MeCo, an adaptive decision-making strategy for external tool use. MeCo quantifies metacognitive scores by capturing high-level cognitive signals in the representation space, guiding when to invoke tools. Notably, MeCo is fine-tuning-free and incurs minimal cost. Experiments across multiple backbone models and benchmarks show that MeCo reliably detects LLMs' internal cognitive signals and significantly improves tool-use decision-making.

cs.CL↗

arXiv2025

Efficient Learning of A Unified Policy For Whole-body Manipulation and Locomotion Skills

Equipping quadruped robots with manipulators provides unique loco-manipulation capabilities, enabling diverse practical applications. This integration creates a more complex system that has increased difficulties in modeling and control. Reinforcement learning (RL) offers a promising solution to address these challenges by learning optimal control policies through interaction. Nevertheless, RL methods often struggle with local optima when exploring large solution spaces for motion and manipulation tasks. To overcome these limitations, we propose a novel approach that integrates an explicit kinematic model of the manipulator into the RL framework. This integration provides feedback on the mapping of the body postures to the manipulator's workspace, guiding the RL exploration process and effectively mitigating the local optima issue. Our algorithm has been successfully deployed on a DeepRobotics X20 quadruped robot equipped with a Unitree Z1 manipulator, and extensive experimental results demonstrate the superior performance of this approach.

cs.RO↗

arXiv2025

LLaVA-KD: A Framework of Distilling Multimodal Large Language Models

The success of Large Language Models (LLMs) has inspired the development of Multimodal Large Language Models (MLLMs) for unified understanding of vision and language. However, the increasing model size and computational complexity of large-scale MLLMs (l-MLLMs) limit their use in resource-constrained scenarios. Although small-scale MLLMs (s-MLLMs) are designed to reduce computational costs, they typically suffer from performance degradation. To mitigate this limitation, we propose a novel LLaVA-KD framework to transfer knowledge from l-MLLMs to s-MLLMs. Specifically, we introduce Multimodal Distillation (MDist) to transfer teacher model's robust representations across both visual and linguistic modalities, and Relation Distillation (RDist) to transfer teacher model's ability to capture visual token relationships. Additionally, we propose a three-stage training scheme to fully exploit the potential of the proposed distillation strategy: 1) Distilled Pre-Training to strengthen the alignment between visual-linguistic representations in s-MLLMs, 2) Supervised Fine-Tuning to equip the s-MLLMs with multimodal understanding capacity, and 3) Distilled Fine-Tuning to refine s-MLLM's knowledge. Our approach significantly improves s-MLLMs performance without altering the model architecture. Extensive experiments and ablation studies validate the effectiveness of each proposed component. Code will be available at https://github.com/Fantasyele/LLaVA-KD.

cs.CV↗

arXiv2025

Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent

Interactive recommendation is a typical information-seeking task that allows users to interactively express their needs through natural language and obtain personalized recommendations. Large language model-powered (LLM-powered) agents have become a new paradigm in interactive recommendations, effectively capturing users' real-time needs and enhancing personalized experiences. However, due to limited planning and generalization capabilities, existing formulations of LLM-powered interactive recommender agents struggle to effectively address diverse and complex user intents, such as intuitive, unrefined, or occasionally ambiguous requests. To tackle this challenge, we propose a novel thought-augmented interactive recommender agent system (TAIRA) that addresses complex user intents through distilled thought patterns. Specifically, TAIRA is designed as an LLM-powered multi-agent system featuring a manager agent that orchestrates recommendation tasks by decomposing user needs and planning subtasks, with its planning capacity strengthened through Thought Pattern Distillation (TPD), a thought-augmentation method that extracts high-level thoughts from the agent's and human experts' experiences. Moreover, we designed a set of user simulation schemes to generate personalized queries of different difficulties and evaluate the recommendations based on specific datasets. Through comprehensive experiments conducted across multiple datasets, TAIRA exhibits significantly enhanced performance compared to existing methods. Notably, TAIRA shows a greater advantage on more challenging tasks while generalizing effectively on novel tasks, further validating its superiority in managing complex user intents within interactive recommendation systems. The code is publicly available at:https://github.com/Alcein/TAIRA.

cs.CL↗

arXiv2025

Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)

Current artificial intelligence models for medical imaging are predominantly single modality and single disease. Attempts to create multimodal and multi-disease models have resulted in inconsistent clinical accuracy. Furthermore, training these models typically requires large, labour-intensive, well-labelled datasets. We developed MerMED-FM, a state-of-the-art multimodal, multi-specialty foundation model trained using self-supervised learning and a memory module. MerMED-FM was trained on 3.3 million medical images from over ten specialties and seven modalities, including computed tomography (CT), chest X-rays (CXR), ultrasound (US), pathology patches, color fundus photography (CFP), optical coherence tomography (OCT) and dermatology images. MerMED-FM was evaluated across multiple diseases and compared against existing foundational models. Strong performance was achieved across all modalities, with AUROCs of 0.988 (OCT); 0.982 (pathology); 0.951 (US); 0.943 (CT); 0.931 (skin); 0.894 (CFP); 0.858 (CXR). MerMED-FM has the potential to be a highly adaptable, versatile, cross-specialty foundation model that enables robust medical imaging interpretation across diverse medical disciplines.

eess.IV↗

arXiv2025

Strain-Induced Non-alter Compensated Magnet and Its Application to Magnetic Tunnel Junction Device Design

The recent proposal of altermagnetism has drawn widespread attention to antiferromagnet (AFM) exhibiting spin splitting, extending beyond the realm of sign-alternating spin splitting in momentum space protected solely by rotational symmetry. Herrin, we propose a shear-strain strategy that enables significant modulation of d-wave altermagnets into an non-alter compensated magnets. A comprehensive analysis combining the magnetic moment compensation characteristics of opposite spin sublattices with the distribution of spin-resolved conduction channels in momentum space under the [001] crystal orientation reveals that shear strain breaks the rotational symmetry of alternatmagnets. To explore the application potential of non-alter compensated magnets, we designe RuO2/TiO2/RuO2 magnetic tunnel junctions (MTJ) with three crystallographic orientations ((001), (110), (100)) and investigated their transport properties under shear strain. This non-alter electronic structure not only enhances the tunneling magnetoresistance (TMR) in spin-split paths of intrinsic RuO2 (226% to 431%) but also enables substantial TMR in spin-degenerate paths (from 0-88%)). Our work provides guidelines for broadening magnetic materials and device platforms.

cond-mat.mtrl-sci↗

arXiv2025

Chemical knowledge-informed framework for privacy-aware retrosynthesis learning

Chemical reaction data is a pivotal asset, driving advances in competitive fields such as pharmaceuticals, materials science, and industrial chemistry. Its proprietary nature renders it sensitive, as it often includes confidential insights and competitive advantages organizations strive to protect. However, in contrast to this need for confidentiality, the current standard training paradigm for machine learning-based retrosynthesis gathers reaction data from multiple sources into one single edge to train prediction models. This paradigm poses considerable privacy risks as it necessitates broad data availability across organizational boundaries and frequent data transmission between entities, potentially exposing proprietary information to unauthorized access or interception during storage and transfer. In the present study, we introduce the chemical knowledge-informed framework (CKIF), a privacy-preserving approach for learning retrosynthesis models. CKIF enables distributed training across multiple chemical organizations without compromising the confidentiality of proprietary reaction data. Instead of gathering raw reaction data, CKIF learns retrosynthesis models through iterative, chemical knowledge-informed aggregation of model parameters. In particular, the chemical properties of predicted reactants are leveraged to quantitatively assess the observable behaviors of individual models, which in turn determines the adaptive weights used for model aggregation. On a variety of reaction datasets, CKIF outperforms several strong baselines by a clear margin.

cs.LG↗

arXiv2025

AdvMIM: Adversarial Masked Image Modeling for Semi-Supervised Medical Image Segmentation

Vision Transformer has recently gained tremendous popularity in medical image segmentation task due to its superior capability in capturing long-range dependencies. However, transformer requires a large amount of labeled data to be effective, which hinders its applicability in annotation scarce semi-supervised learning scenario where only limited labeled data is available. State-of-the-art semi-supervised learning methods propose combinatorial CNN-Transformer learning to cross teach a transformer with a convolutional neural network, which achieves promising results. However, it remains a challenging task to effectively train the transformer with limited labeled data. In this paper, we propose an adversarial masked image modeling method to fully unleash the potential of transformer for semi-supervised medical image segmentation. The key challenge in semi-supervised learning with transformer lies in the lack of sufficient supervision signal. To this end, we propose to construct an auxiliary masked domain from original domain with masked image modeling and train the transformer to predict the entire segmentation mask with masked inputs to increase supervision signal. We leverage the original labels from labeled data and pseudo-labels from unlabeled data to learn the masked domain. To further benefit the original domain from masked domain, we provide a theoretical analysis of our method from a multi-domain learning perspective and devise a novel adversarial training loss to reduce the domain gap between the original and masked domain, which boosts semi-supervised learning performance. We also extend adversarial masked image modeling to CNN network. Extensive experiments on three public medical image segmentation datasets demonstrate the effectiveness of our method, where our method outperforms existing methods significantly. Our code is publicly available at https://github.com/zlheui/AdvMIM.

cs.CV↗

arXiv2025

Stepping Out of Similar Semantic Space for Open-Vocabulary Segmentation

Open-vocabulary segmentation aims to achieve segmentation of arbitrary categories given unlimited text inputs as guidance. To achieve this, recent works have focused on developing various technical routes to exploit the potential of large-scale pre-trained vision-language models and have made significant progress on existing benchmarks. However, we find that existing test sets are limited in measuring the models' comprehension of ``open-vocabulary" concepts, as their semantic space closely resembles the training space, even with many overlapping categories. To this end, we present a new benchmark named OpenBench that differs significantly from the training semantics. It is designed to better assess the model's ability to understand and segment a wide range of real-world concepts. When testing existing methods on OpenBench, we find that their performance diverges from the conclusions drawn on existing test sets. In addition, we propose a method named OVSNet to improve the segmentation performance for diverse and open scenarios. Through elaborate fusion of heterogeneous features and cost-free expansion of the training space, OVSNet achieves state-of-the-art results on both existing datasets and our proposed OpenBench. Corresponding analysis demonstrate the soundness and effectiveness of our proposed benchmark and method.

cs.CV↗

arXiv2025

Computational Discovery of Metastable NaMnO$_2$ Polymorphs as High-Performance Cathodes with Ultralow Na$^+$ Migration Barriers

Using an ab initio evolutionary algorithm combined with first-principles calculations, two metastable NaMnO$_2$ polymorphs, $I4_1/amd$ and Cmcm, are identified as promising cathode materials for sodium-ion batteries. Both phases exhibit excellent thermodynamic stability, lying within 35~meV/atom of the ground-state \textit{Pmmn} phase across 0--50~GPa, and are dynamically and thermally stable under ambient conditions following high-pressure synthesis, as confirmed by phonon and ab initio molecular dynamics simulations. During desodiation, a Jahn--Teller-induced magnetic transition enhances Mn--O hybridization, reduces the bandgap, and promotes robust charge compensation and oxygen retention. Remarkably, the Cmcm phase achieves record-low Na$^+$ migration barriers (0.39~eV at high Na concentration; 0.27~eV at low concentration), representing 47\% and 36\% reductions respectively compared to conventional $C2/m$, while delivering a higher average voltage (3.19~V vs 2.88~V). The $I4_1/amd$ phase exhibits concentration-dependent diffusion with a low-energy pathway (0.38~eV) and maintains competitive voltage (2.94~V). These findings suggest that metastable NaMnO$_2$ polymorphs may offer viable alternatives to conventional cathode materials, particularly where fast ionic conduction is required.

cond-mat.mtrl-sci↗

arXiv2025

Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning

Test-time scaling, which is also often referred to as slow-thinking, has been demonstrated to enhance multi-step reasoning in large language models (LLMs). However, despite its widespread utilization, the mechanisms underlying slow-thinking methods remain poorly understood. This paper explores the mechanisms of external slow-thinking from a theoretical standpoint. We begin by examining the snowball error effect within the LLM reasoning process and connect it to the likelihood of correct reasoning using information theory. Building on this, we show that external slow-thinking methods can be interpreted as strategies to mitigate the error probability. We further provide a comparative analysis of popular external slow-thinking approaches, ranging from simple to complex, highlighting their differences and interrelationships. Our findings suggest that the efficacy of these methods is not primarily determined by the specific framework employed, and that expanding the search scope or the model's internal reasoning capacity may yield more sustained improvements in the long term. We open-source our code at https://github.com/ZyGan1999/Snowball-Errors-and-Probability.

cs.AI↗

arXiv2025

Pieceformer: Similarity-Driven Knowledge Transfer via Scalable Graph Transformer in VLSI

Accurate graph similarity is critical for knowledge transfer in VLSI design, enabling the reuse of prior solutions to reduce engineering effort and turnaround time. We propose Pieceformer, a scalable, self-supervised similarity assessment framework, equipped with a hybrid message-passing and graph transformer encoder. To address transformer scalability, we incorporate a linear transformer backbone and introduce a partitioned training pipeline for efficient memory and parallelism management. Evaluations on synthetic and real-world CircuitNet datasets show that Pieceformer reduces mean absolute error (MAE) by 24.9% over the baseline and is the only method to correctly cluster all real-world design groups. We further demonstrate the practical usage of our model through a case study on a partitioning task, achieving up to 89% runtime reduction. These results validate the framework's effectiveness for scalable, unbiased design reuse in modern VLSI systems.

cs.LG↗

arXiv2025

DreamLight: Towards Harmonious and Consistent Image Relighting

We introduce a model named DreamLight for universal image relighting in this work, which can seamlessly composite subjects into a new background while maintaining aesthetic uniformity in terms of lighting and color tone. The background can be specified by natural images (image-based relighting) or generated from unlimited text prompts (text-based relighting). Existing studies primarily focus on image-based relighting, while with scant exploration into text-based scenarios. Some works employ intricate disentanglement pipeline designs relying on environment maps to provide relevant information, which grapples with the expensive data cost required for intrinsic decomposition and light source. Other methods take this task as an image translation problem and perform pixel-level transformation with autoencoder architecture. While these methods have achieved decent harmonization effects, they struggle to generate realistic and natural light interaction effects between the foreground and background. To alleviate these challenges, we reorganize the input data into a unified format and leverage the semantic prior provided by the pretrained diffusion model to facilitate the generation of natural results. Moreover, we propose a Position-Guided Light Adapter (PGLA) that condenses light information from different directions in the background into designed light query embeddings, and modulates the foreground with direction-biased masked attention. In addition, we present a post-processing module named Spectral Foreground Fixer (SFF) to adaptively reorganize different frequency components of subject and relighted background, which helps enhance the consistency of foreground appearance. Extensive comparisons and user study demonstrate that our DreamLight achieves remarkable relighting performance.

cs.CV↗