arXiv subjects

Yan Zhang

Publications and source records attributed to Yan Zhang.

At least 163 records · Page 9Linked to original sources

arXiv2025

LTD-Bench: Evaluating Large Language Models by Letting Them Draw

Current evaluation paradigms for large language models (LLMs) represent a critical blind spot in AI research--relying on opaque numerical metrics that conceal fundamental limitations in spatial reasoning while providing no intuitive understanding of model capabilities. This deficiency creates a dangerous disconnect between reported performance and practical abilities, particularly for applications requiring physical world understanding. We introduce LTD-Bench, a breakthrough benchmark that transforms LLM evaluation from abstract scores to directly observable visual outputs by requiring models to generate drawings through dot matrices or executable code. This approach makes spatial reasoning limitations immediately apparent even to non-experts, bridging the fundamental gap between statistical performance and intuitive assessment. LTD-Bench implements a comprehensive methodology with complementary generation tasks (testing spatial imagination) and recognition tasks (assessing spatial perception) across three progressively challenging difficulty levels, methodically evaluating both directions of the critical language-spatial mapping. Our extensive experiments with state-of-the-art models expose an alarming capability gap: even LLMs achieving impressive results on traditional benchmarks demonstrate profound deficiencies in establishing bidirectional mappings between language and spatial concept--a fundamental limitation that undermines their potential as genuine world models. Furthermore, LTD-Bench's visual outputs enable powerful diagnostic analysis, offering a potential approach to investigate model similarity.

cs.CL↗

arXiv2025

VidText: Towards Comprehensive Evaluation for Video Text Understanding

Visual texts embedded in videos carry rich semantic information, which is crucial for both holistic video understanding and fine-grained reasoning about local human actions. However, existing video understanding benchmarks largely overlook textual information, while OCR-specific benchmarks are constrained to static images, limiting their ability to capture the interaction between text and dynamic visual contexts. To address this gap, we propose VidText, a new benchmark designed for comprehensive and in-depth evaluation of video text understanding. VidText offers the following key features: 1) It covers a wide range of real-world scenarios and supports multilingual content, encompassing diverse settings where video text naturally appears. 2) It introduces a hierarchical evaluation framework with video-level, clip-level, and instance-level tasks, enabling assessment of both global summarization and local retrieval capabilities. 3) The benchmark also introduces a set of paired perception reasoning tasks, ranging from visual text perception to cross-modal reasoning between textual and visual information. Extensive experiments on 18 state-of-the-art Large Multimodal Models (LMMs) reveal that current models struggle across most tasks, with significant room for improvement. Further analysis highlights the impact of both model-intrinsic factors, such as input resolution and OCR capability, and external factors, including the use of auxiliary information and Chain-of-Thought reasoning strategies. We hope VidText will fill the current gap in video understanding benchmarks and serve as a foundation for future research on multimodal reasoning with video text in dynamic environments.

cs.CV↗

arXiv2025

Structurally Refined Graph Transformer for Multimodal Recommendation

Multimodal recommendation systems utilize various types of information, including images and text, to enhance the effectiveness of recommendations. The key challenge is predicting user purchasing behavior from the available data. Current recommendation models prioritize extracting multimodal information while neglecting the distinction between redundant and valuable data. They also rely heavily on a single semantic framework (e.g., local or global semantics), resulting in an incomplete or biased representation of user preferences, particularly those less expressed in prior interactions. Furthermore, these approaches fail to capture the complex interactions between users and items, limiting the model's ability to meet diverse users. To address these challenges, we present SRGFormer, a structurally optimized multimodal recommendation model. By modifying the transformer for better integration into our model, we capture the overall behavior patterns of users. Then, we enhance structural information by embedding multimodal information into a hypergraph structure to aid in learning the local structures between users and items. Meanwhile, applying self-supervised tasks to user-item collaborative signals enhances the integration of multimodal information, thereby revealing the representational features inherent to the data's modality. Extensive experiments on three public datasets reveal that SRGFormer surpasses previous benchmark models, achieving an average performance improvement of 4.47 percent on the Sports dataset. The code is publicly available online.

cs.IR↗

arXiv2025

Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracy

We introduce Long-VITA, a simple yet effective large multi-modal model for long-context visual-language understanding tasks. It is adept at concurrently processing and analyzing modalities of image, video, and text over 4K frames or 1M tokens while delivering advanced performances on short-context multi-modal tasks. We propose an effective multi-modal training schema that starts with large language models and proceeds through vision-language alignment, general knowledge learning, and two sequential stages of long-sequence fine-tuning. We further implement context-parallelism distributed inference and logits-masked language modeling head to scale Long-VITA to infinitely long inputs of images and texts during model inference. Regarding training data, Long-VITA is built on a mix of 17M samples from public datasets only and demonstrates state-of-the-art performance on various multi-modal benchmarks, compared against recent cutting-edge models with internal data. Long-VITA is fully open-source and reproducible.. By leveraging our inference designs, Long-VITA models achieve a remarkable 2x prefill speedup and 4x context length extension in a single node with 8 GPUs. We hope Long-VITA can serve as a competitive baseline and offer valuable insights for the open-source community in advancing long-context multi-modal understanding.

cs.CV↗

arXiv2025

Common-Individual Embedding for Dynamic Networks with Temporal Group Structure

We propose STANE (Shared and Time-specific Adaptive Network Embedding), a new joint embedding framework for dynamic networks that captures both stable global structures and localized temporal variations. To further improve the model's adaptability to transient changes, we introduce Sparse STANE, which models time-specific changes as sparse perturbations, thereby improving interpretability. Unlike existing methods that either overlook cross-time similarities or enforce overly smooth evolution, Sparse STANE integrates temporal clustering with sparse deviation modeling to strike a flexible balance between persistence and change. We also provide non-asymptotic error guarantees of embeddings and show that our estimator can reliably identify changed node pairs when deviations are sparse. On synthetic and real-world political conflict networks, STANE and its extensions improve temporal clustering accuracy and structural recovery, outperforming state-of-the-art baselines. These results highlight the potential of STANE in applications such as international relations modeling, where both persistent and transient connections matter. Our findings underscore the power of structured dynamic embeddings for revealing interpretable patterns in network evolution.

stat.ME↗

arXiv2025

Chern-Selective multi-valley Flat Bands in Twisted Mono-Bilayer and Mono-Trilayer MoTe$_2$

The interplay between moiré flat bands originating from different valleys can give rise to a variety of exotic quantum phases. In this work, we investigate the electronic properties of twisted mono-bilayer (A-AB) and mono-trilayer (A-ABA) MoTe$_2$ using first-principles calculations and continuum models. Unlike previous studies on twisted bilayer systems, in which low-energy flat bands originate solely from the $K/K'$ valleys, in A-AB and A-ABA twisted MoTe$_2$ (\tmt) the moiré bands at low energies arise from both the $Γ$ and $K/K'$ valleys, with spin Chern numbers $C_s=0$ (for $Γ$) and $C_{\uparrow/\downarrow}=\pm1$ (for $K/K'$), respectively. We show that the multi-valley moiré flat bands are governed by interlayer-hybridization effects, and that different stacking configurations and thicknesses tune the relative energy alignment between the $Γ$ and $K$ valley moiré flat bands. By constructing valley-resolved continuum models and performing Wannierization for the low-energy moiré bands, we further uncover that the Berry curvature and quantum metric distributions can be effectively tuned by the layer number and stacking configuration. Unlike other moiré systems, where only one kind of valley influenced the low energy physics, the simultaneous appearance of two distinct types of valleys, with different symmetries, establish A-AB and A-ABA \tmt\ as ideal platforms for studying layer-controlled multi-valley physics.

cond-mat.mtrl-sci↗

arXiv2025

Contrastive Local Manifold Learning for No-Reference Image Quality Assessment

Image Quality Assessment (IQA) methods typically overlook local manifold structures, leading to compromised discriminative capabilities in perceptual quality evaluation. To address this limitation, we present LML-IQA, an innovative no-reference IQA (NR-IQA) approach that leverages a combination of local manifold learning and contrastive learning. Our approach first extracts multiple patches from each image and identifies the most visually salient region. This salient patch serves as a positive sample for contrastive learning, while other patches from the same image are treated as intra-class negatives to preserve local distinctiveness. Patches from different images also act as inter-class negatives to enhance feature separation. Additionally, we introduce a mutual learning strategy to improve the model's ability to recognize and prioritize visually important regions. Comprehensive experiments across eight benchmark datasets demonstrate significant performance gains over state-of-the-art methods, achieving a PLCC of 0.942 on TID2013 (compared to 0.908) and 0.977 on CSIQ (compared to 0.965).

cs.CV↗

arXiv2025

Integrated Analysis for Electronic Health Records with Structured and Sporadic Missingness

Objectives: We propose a novel imputation method tailored for Electronic Health Records (EHRs) with structured and sporadic missingness. Such missingness frequently arises in the integration of heterogeneous EHR datasets for downstream clinical applications. By addressing these gaps, our method provides a practical solution for integrated analysis, enhancing data utility and advancing the understanding of population health. Materials and Methods: We begin by demonstrating structured and sporadic missing mechanisms in the integrated analysis of EHR data. Following this, we introduce a novel imputation framework, Macomss, specifically designed to handle structurally and heterogeneously occurring missing data. We establish theoretical guarantees for Macomss, ensuring its robustness in preserving the integrity and reliability of integrated analyses. To assess its empirical performance, we conduct extensive simulation studies that replicate the complex missingness patterns observed in real-world EHR systems, complemented by validation using EHR datasets from the Duke University Health System (DUHS). Results: Simulation studies show that our approach consistently outperforms existing imputation methods. Using datasets from three hospitals within DUHS, Macomss achieves the lowest imputation errors for missing data in most cases and provides superior or comparable downstream prediction performance compared to benchmark methods. Conclusions: We provide a theoretically guaranteed and practically meaningful method for imputing structured and sporadic missing data, enabling accurate and reliable integrated analysis across multiple EHR datasets. The proposed approach holds significant potential for advancing research in population health.

stat.AP↗

arXiv2025

Selective high-order topological states and tunable chiral emission in atomic metasurfaces

Atomic metasurfaces (AMs) provide a powerful nanophotonic platform for integrating topological effects into quantum many-body systems. In this Letter, we investigate the quantum optical and topological properties of a two-dimensional Kagome AM, going beyond the tight-binding approximation and incorporating all-to-all interactions. We reveal selective higher-order topological states with a unique dynamical ``chasing" behavior, protected by a generalized chiral symmetry and enabling efficient topological directional transfer. By introducing an impurity atom -- a giant atom -- coupled to all array atoms, we observe chiral emission patterns strongly dependent on the atomic polarization. This nonlocal coupling structure allows exploration of self-interference effects at subwavelength scales. Our findings establish AMs as a versatile platform for engineering tunable topological states and chiral quantum optical phenomena, with potential applications in customized light sources and photonic devices.

quant-ph↗

arXiv2025

When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding

Large Multimodal Models (LMMs) have achieved impressive progress in visual perception and reasoning. However, when confronted with visually ambiguous or non-semantic scene text, they often struggle to accurately spot and understand the content, frequently generating semantically plausible yet visually incorrect answers, which we refer to as semantic hallucination. In this work, we investigate the underlying causes of semantic hallucination and identify a key finding: Transformer layers in LLM with stronger attention focus on scene text regions are less prone to producing semantic hallucinations. Thus, we propose a training-free semantic hallucination mitigation framework comprising two key components: (1) ZoomText, a coarse-to-fine strategy that identifies potential text regions without external detectors; and (2) Grounded Layer Correction, which adaptively leverages the internal representations from layers less prone to hallucination to guide decoding, correcting hallucinated outputs for non-semantic samples while preserving the semantics of meaningful ones. To enable rigorous evaluation, we introduce TextHalu-Bench, a benchmark of 1,740 samples spanning both semantic and non-semantic cases, with manually curated question answer pairs designed to probe model hallucinations. Extensive experiments demonstrate that our method not only effectively mitigates semantic hallucination but also achieves strong performance on public benchmarks for scene text spotting and understanding.

cs.CV↗

arXiv2025

"When I lost it, they dragged me out": How Care Encounters Empower Marginalized Young Adults' Aspiration and Mental Health Care-Seeking

Mental health care-seeking among marginalized young adults has received limited attention in CSCW research. Through in-depth interviews and visual elicitation methods with 18 diverse U.S. participants, our study reveals how marginalized identities shape mental health care-seeking journeys, often characterized by low aspirations and passive care-seeking influenced by lived experiences of marginalization. However, we found the transformative function of "care encounters" - serendipitous interactions with mental health resources that occur when individuals are not actively seeking support. These encounters serve as critical turning points, catalyzing shifts in aspiration and enabling more proactive care-seeking behaviors. Our analysis identifies both the infrastructural conditions that enable transformative care encounters and the aspiration breakdowns that impede care-seeking processes. This work makes conceptual contributions by supplementing traditional motivation-based care-seeking models with a reconceptualization of "care encounters" that accounts for the infrastructural and serendipitous nature of mental health access. We advance understanding of how marginalized identity uniquely influences care-seeking behaviors while providing actionable design implications for embedding technology-mediated "care encounters" into socio-technical interventions that can better support mental health care access for vulnerable populations.

cs.HC↗

arXiv2025

Experience Paper: Adopting Activity Recognition in On-demand Food Delivery Business

This paper presents the first nationwide deployment of human activity recognition (HAR) technology in the on-demand food delivery industry. We successfully adapted the state-of-the-art LIMU-BERT foundation model to the delivery platform. Spanning three phases over two years, the deployment progresses from a feasibility study in Yangzhou City to nationwide adoption involving 500,000 couriers across 367 cities in China. The adoption enables a series of downstream applications, and large-scale tests demonstrate its significant operational and economic benefits, showcasing the transformative potential of HAR technology in real-world applications. Additionally, we share lessons learned from this deployment and open-source our LIMU-BERT pretrained with millions of hours of sensor data.

cs.AI↗

arXiv2025

Beyond Heuristics: Globally Optimal Configuration of Implicit Neural Representations

Implicit Neural Representations (INRs) have emerged as a transformative paradigm in signal processing and computer vision, excelling in tasks from image reconstruction to 3D shape modeling. Yet their effectiveness is fundamentally limited by the absence of principled strategies for optimal configuration - spanning activation selection, initialization scales, layer-wise adaptation, and their intricate interdependencies. These choices dictate performance, stability, and generalization, but current practice relies on ad-hoc heuristics, brute-force grid searches, or task-specific tuning, often leading to inconsistent results across modalities. This work introduces OptiINR, the first unified framework that formulates INR configuration as a rigorous optimization problem. Leveraging Bayesian optimization, OptiINR efficiently explores the joint space of discrete activation families - such as sinusoidal (SIREN), wavelet-based (WIRE), and variable-periodic (FINER) - and their associated continuous initialization parameters. This systematic approach replaces fragmented manual tuning with a coherent, data-driven optimization process. By delivering globally optimal configurations, OptiINR establishes a principled foundation for INR design, consistently maximizing performance across diverse signal processing applications.

cs.LG↗

arXiv2025

SCRA-VQA: Summarized Caption-Rerank for Augmented Large Language Models in Visual Question Answering

Acquiring high-quality knowledge is a central focus in Knowledge-Based Visual Question Answering (KB-VQA). Recent methods use large language models (LLMs) as knowledge engines for answering. These methods generally employ image captions as visual text descriptions to assist LLMs in interpreting images. However, the captions frequently include excessive noise irrelevant to the question, and LLMs generally do not comprehend VQA tasks, limiting their reasoning capabilities. To address this issue, we propose the Summarized Caption-Rerank Augmented VQA (SCRA-VQA), which employs a pre-trained visual language model to convert images into captions. Moreover, SCRA-VQA generates contextual examples for the captions while simultaneously summarizing and reordering them to exclude unrelated information. The caption-rerank process enables LLMs to understand the image information and questions better, thus enhancing the model's reasoning ability and task adaptability without expensive end-to-end training. Based on an LLM with 6.7B parameters, SCRA-VQA performs excellently on two challenging knowledge-based VQA datasets: OK-VQA and A-OKVQA, achieving accuracies of 38.8% and 34.6%. Our code is available at https://github.com/HubuKG/SCRA-VQA.

cs.CV↗

arXiv2025

RecIS: Sparse to Dense, A Unified Training Framework for Recommendation Models

In this paper, we propose RecIS, a unified Sparse-Dense training framework designed to achieve two primary goals: 1. Unified Framework To create a Unified sparse-dense training framework based on the PyTorch ecosystem that meets the training needs of industrial-grade recommendation models that integrated with large models. 2.System Optimization To optimize the sparse component, offering superior efficiency over the TensorFlow-based recommendation models. The dense component, meanwhile, leverages existing optimization technologies within the PyTorch ecosystem. Currently, RecIS is being used in Alibaba for numerous large-model enhanced recommendation training tasks, and some traditional sparse models have also begun training in it.

cs.IR↗

arXiv2025

Integrating Object Interaction Self-Attention and GAN-Based Debiasing for Visual Question Answering

Visual Question Answering (VQA) presents a unique challenge by requiring models to understand and reason about visual content to answer questions accurately. Existing VQA models often struggle with biases introduced by the training data, leading to over-reliance on superficial patterns and inadequate generalization to diverse questions and images. This paper presents a novel model, IOG-VQA, which integrates Object Interaction Self-Attention and GAN-Based Debiasing to enhance VQA model performance. The self-attention mechanism allows our model to capture complex interactions between objects within an image, providing a more comprehensive understanding of the visual context. Meanwhile, the GAN-based debiasing framework generates unbiased data distributions, helping the model to learn more robust and generalizable features. By leveraging these two components, IOG-VQA effectively combines visual and textual information to address the inherent biases in VQA datasets. Extensive experiments on the VQA-CP v1 and VQA-CP v2 datasets demonstrate that our model shows excellent performance compared with the existing methods, particularly in handling biased and imbalanced data distributions highlighting the importance of addressing both object interactions and dataset biases in advancing VQA tasks. Our code is available at https://github.com/HubuKG/IOG-VQA.

cs.CV↗

arXiv2025

A statistical study of type II outbursts of XRPs: Brighter accreting pulsars rotate faster

X-ray pulsars (XRPs) consist of a magnetized neutron star (NS) and an optical donor star. The NS accretes matter from the donor star producing pulsed X-ray emission. In most cases the donor stars are Be stars, and accretion is episodic, that is, the NSs are generally X-ray dim, but occasionally experience outbursts. Here, we carry out a statistical study with the X-ray monitoring data, and obtain strong correlations between the spin periods of the NSs and the outburst parameters for the first time. We show that XRPs containing faster rotating NSs tend to display more violent eruptions. In addition, pulsating ultraluminous X-ray sources in nearby galaxies follow the similar relationship. We demonstrate that most of these systems are close to the spin equilibrium, and that brighter pulsars have acquired more angular momentum by accreting matter from their companion stars, resulting in faster rotating NSs.

astro-ph.HE↗

arXiv2025

PPORLD-EDNetLDCT: A Proximal Policy Optimization-Based Reinforcement Learning Framework for Adaptive Low-Dose CT Denoising

Low-dose computed tomography (LDCT) is critical for minimizing radiation exposure, but it often leads to increased noise and reduced image quality. Traditional denoising methods, such as iterative optimization or supervised learning, often fail to preserve image quality. To address these challenges, we introduce PPORLD-EDNetLDCT, a reinforcement learning-based (RL) approach with Encoder-Decoder for LDCT. Our method utilizes a dynamic RL-based approach in which an advanced posterior policy optimization (PPO) algorithm is used to optimize denoising policies in real time, based on image quality feedback, trained via a custom gym environment. The experimental results on the low dose CT image and projection dataset demonstrate that the proposed PPORLD-EDNetLDCT model outperforms traditional denoising techniques and other DL-based methods, achieving a peak signal-to-noise ratio of 41.87, a structural similarity index measure of 0.9814 and a root mean squared error of 0.00236. Moreover, in NIH-AAPM-Mayo Clinic Low Dose CT Challenge dataset our method achieved a PSNR of 41.52, SSIM of 0.9723 and RMSE of 0.0051. Furthermore, we validated the quality of denoising using a classification task in the COVID-19 LDCT dataset, where the images processed by our method improved the classification accuracy to 94%, achieving 4% higher accuracy compared to denoising without RL-based denoising.

cs.CV↗