arXiv subjects

Fan Zhang

Publications and source records attributed to Fan Zhang.

At least 343 records · Page 19Linked to original sources

arXiv2025

RISE: Enhancing VLM Image Annotation with Self-Supervised Reasoning

Vision-Language Models (VLMs) struggle with complex image annotation tasks, such as emotion classification and context-driven object detection, which demand sophisticated reasoning. Standard Supervised Fine-Tuning (SFT) focuses solely on annotation outcomes, ignoring underlying rationales, while Visual Reinforcement Fine-Tuning (Visual-RFT) produces inconsistent Chains of Thought (CoTs) due to the absence of high-quality, verified CoTs during pre-training. We introduce RISE (Reason-Inspire-Strengthen-Expertise), a two-stage framework to overcome these limitations. In the Reason stage (RISE-CoT), a reinforcement learning-driven "annotation-reasoning-annotation" closed-loop generates visually grounded, logically consistent CoTs by verifying their ability to reconstruct original annotations without direct leakage. The Inspire and Strengthen stage (RISE-R1) leverages a high-quality CoT subset, filtered by RISE-CoT rewards, for supervised fine-tuning, followed by reinforcement fine-tuning to produce interpretable reasoning and accurate annotations, achieving Expertise in complex visual tasks. Evaluated on complex and simple image annotation tasks, RISE-trained Qwen2-VL-2B outperforms SFT and Visual-RFT, achieving robust performance and enhanced explainability. RISE offers a self-supervised solution for advancing VLM reasoning without requiring manually annotated CoTs.Code and resources are available at: https://github.com/HSH55/RISE.

cs.LG↗

arXiv2025

A General Nonlinear Model for Arbitrary Modulation Formats in the Presence of Inter-Channel Simulated Raman Scattering

The four-dimensional nonlinear model is extended to include the inter-channel stimulated Raman scattering, enabling accurate prediction of dual-polarization four-dimensional modulation formats and probabilistically shaped constellations in high-dispersion regimes. The proposed model is validated via comparisons with the split-step Fourier method and enhanced Gaussian noise model.

eess.SP↗

arXiv2025

TractGraphFormer: Anatomically Informed Hybrid Graph CNN-Transformer Network for Interpretable Sex and Age Prediction from Diffusion MRI Tractography

The relationship between brain connections and non-imaging phenotypes is increasingly studied using deep neural networks. However, the local and global properties of brain white matter networks are often overlooked in convolutional network design. We introduce TractGraphFormer, a hybrid Graph CNN-Transformer deep learning framework tailored for diffusion MRI tractography. This model leverages local anatomical characteristics and global feature dependencies of white matter structures. The Graph CNN module captures white matter geometry and grey matter connectivity to aggregate local features from anatomically similar white matter connections, while the Transformer module uses self-attention to enhance global information learning. Additionally, TractGraphFormer includes an attention module for interpreting predictive white matter connections. We apply TractGraphFormer to tasks of sex and age prediction. TractGraphFormer shows strong performance in large datasets of children (n=9345) and young adults (n=1065). Overall, our approach suggests that widespread connections in the WM are predictive of the sex and age of an individual. For each prediction task, consistent predictive anatomical tracts are identified across the two datasets. The proposed approach highlights the potential of integrating local anatomical information and global feature dependencies to improve prediction performance in machine learning with diffusion MRI tractography.

cs.CV↗

arXiv2025

The enhanced X-ray Timing and Polarimetry mission -- eXTP for launch in 2030

In this paper we present the current status of the enhanced X-ray Timing and Polarimetry mission, which has been fully approved for launch in 2030. eXTP is a space science mission designed to study fundamental physics under extreme conditions of matter density, gravity, and magnetism. The mission aims at determining the equation of state of matter at supra-nuclear density, measuring the effects of quantum electro-dynamics, and understanding the dynamics of matter in strong-field gravity. In addition to investigating fundamental physics, the eXTP mission is poised to become a leading observatory for time-domain and multi-messenger astronomy in the 2030's, as well as providing observations of unprecedented quality on a variety of galactic and extragalactic objects. After briefly introducing the history and a summary of the scientific objectives of the eXTP mission, this paper presents a comprehensive overview of: 1) the cutting-edge technology, technical specifications, and anticipated performance of the mission's scientific instruments; 2) the full mission profile, encompassing spacecraft design, operational capabilities, and ground segment infrastructure.

astro-ph.HE↗

arXiv2025

Material Experience: An Evaluation Model for Creative Materials Based on Visual-Tactile Sensory Properties

This study adopts a design-oriented approach to integrate traditional braids with commonly used matrix materials, developing creative materials with different sensory properties by altering matrix material types and braid patterns. Based on these creative materials, a quantitative and structured model is proposed to assist designers understanding the material experience process and guide material selection by analyzing the relationship between material properties and sensory perception. Specifically, participants evaluated the creative materials under visual-tactile conditions using a 7-point semantic differential (SD) scale. Correlation analysis was performed to explore the data. The main and interaction effects of matrix materials and braid patterns on impression evaluation were analyzed using two-way analysis of variance (ANOVA). A structural equation model (SEM) was constructed based on exploratory factor analysis (EFA), and path coefficients were computed to assess the relative importance of material properties in determining material attractiveness. The results show that, compared to braids, the creative materials resulted in significant changes in impression evaluation. Furthermore, the creative materials can be understood through intrinsic, aesthetic, and physical properties, with their standardized regression coefficients for material attractiveness of 0.486, 0.650, and 0.103, respectively. These properties are interrelated and under their combined influence affect the attractiveness of the material. Therefore, designers should consider utilizing these relationships to enhance sensory experience in order to achieve design objectives. Moreover, designers should also consider balancing technology and experience, using materials according to the principle of "form follows function".

cs.HC↗

arXiv2025

Micro-Expression Recognition via Fine-Grained Dynamic Perception

Facial micro-expression recognition (MER) is a challenging task, due to the transience, subtlety, and dynamics of micro-expressions (MEs). Most existing methods resort to hand-crafted features or deep networks, in which the former often additionally requires key frames, and the latter suffers from small-scale and low-diversity training data. In this paper, we develop a novel fine-grained dynamic perception (FDP) framework for MER. We propose to rank frame-level features of a sequence of raw frames in chronological order, in which the rank process encodes the dynamic information of both ME appearances and motions. Specifically, a novel local-global feature-aware transformer is proposed for frame representation learning. A rank scorer is further adopted to calculate rank scores of each frame-level feature. Afterwards, the rank features from rank scorer are pooled in temporal dimension to capture dynamic representation. Finally, the dynamic representation is shared by a MER module and a dynamic image construction module, in which the former predicts the ME category, and the latter uses an encoder-decoder structure to construct the dynamic image. The design of dynamic image construction task is beneficial for capturing facial subtle actions associated with MEs and alleviating the data scarcity issue. Extensive experiments show that our method (i) significantly outperforms the state-of-the-art MER methods, and (ii) works well for dynamic image construction. Particularly, our FDP improves by 4.05%, 2.50%, 7.71%, and 2.11% over the previous best results in terms of F1-score on the CASME II, SAMM, CAS(ME)^2, and CAS(ME)^3 datasets, respectively. The code is available at https://github.com/CYF-cuber/FDP.

cs.CV↗

arXiv2025

Towards Interpretable Geo-localization: a Concept-Aware Global Image-GPS Alignment Framework

Worldwide geo-localization involves determining the exact geographic location of images captured globally, typically guided by geographic cues such as climate, landmarks, and architectural styles. Despite advancements in geo-localization models like GeoCLIP, which leverages images and location alignment via contrastive learning for accurate predictions, the interpretability of these models remains insufficiently explored. Current concept-based interpretability methods fail to align effectively with Geo-alignment image-location embedding objectives, resulting in suboptimal interpretability and performance. To address this gap, we propose a novel framework integrating global geo-localization with concept bottlenecks. Our method inserts a Concept-Aware Alignment Module that jointly projects image and location embeddings onto a shared bank of geographic concepts (e.g., tropical climate, mountain, cathedral) and minimizes a concept-level loss, enhancing alignment in a concept-specific subspace and enabling robust interpretability. To our knowledge, this is the first work to introduce interpretability into geo-localization. Extensive experiments demonstrate that our approach surpasses GeoCLIP in geo-localization accuracy and boosts performance across diverse geospatial prediction tasks, revealing richer semantic insights into geographic decision-making processes.

cs.CV↗

arXiv2025

HDN: Hybrid Deep-Learning and Non-Line-of-Sight Reconstruction Framework for Transcranial Photoacoustic Imaging of Human Brain

Photoacoustic imaging combines the high contrast of optical imaging with the deep penetration depth of ultrasonic imaging, showing great potential in cerebrovascular disease detection. However, the ultrasonic wave suffers strong attenuation and multi-scattering when it passes through the skull tissue, resulting in the distortion of the collected photoacoustic signal. In this paper, inspired by the principles of deep learning and non-line-of-sight imaging, we propose an image reconstruction framework named HDN (Hybrid Deep-learning and Non-line-of-sight), which consists of the signal extraction part and difference utilization part. The signal extraction part is used to correct the distorted signal and reconstruct an initial image. The difference utilization part is used to make further use of the signal difference between the distorted signal and corrected signal, reconstructing the residual image between the initial image and the target image. The test results on a photoacoustic digital brain simulation dataset show that compared with the traditional method (delay-and-sum) and deep-learning-based method (UNet), the HDN achieved superior performance in both signal correction and image reconstruction. Specifically for the structural similarity index, the HDN reached 0.661 in imaging results, compared to 0.157 for the delay-and-sum method and 0.305 for the deep-learning-based method.

physics.med-ph↗

arXiv2025

DeepNuParc: A Novel Deep Clustering Framework for Fine-scale Parcellation of Brain Nuclei Using Diffusion MRI Tractography

Brain nuclei are clusters of anatomically distinct neurons that serve as important hubs for processing and relaying information in various neural circuits. Fine-scale parcellation of the brain nuclei is vital for a comprehensive understanding of its anatomico-functional correlations. Diffusion MRI tractography is an advanced imaging technique that can estimate the brain's white matter structural connectivity to potentially reveal the topography of the nuclei of interest for studying its subdivisions. In this work, we present a deep clustering pipeline, namely DeepNuParc, to perform automated, fine-scale parcellation of brain nuclei using diffusion MRI tractography. First, we incorporate a newly proposed deep learning approach to enable accurate segmentation of the nuclei of interest directly on the dMRI data. Next, we design a novel streamline clustering-based structural connectivity feature for a robust representation of voxels within the nuclei. Finally, we improve the popular joint dimensionality reduction and k-means clustering approach to enable nuclei parcellation at a finer scale. We demonstrate DeepNuParc on two important brain structures, i.e. the amygdala and the thalamus, that are known to have multiple anatomically and functionally distinct nuclei subdivisions. Experimental results show that DeepNuParc enables consistent parcellation of the nuclei into multiple parcels across multiple subjects and achieves good correspondence with the widely used coarse-scale atlases. Our codes are available at https://github.com/HarlandZZC/deep_nuclei_parcellation.

eess.IV↗

arXiv2025

PreGenie: An Agentic Framework for High-quality Visual Presentation Generation

Visual presentations are vital for effective communication. Early attempts to automate their creation using deep learning often faced issues such as poorly organized layouts, inaccurate text summarization, and a lack of image understanding, leading to mismatched visuals and text. These limitations restrict their application in formal contexts like business and scientific research. To address these challenges, we propose PreGenie, an agentic and modular framework powered by multimodal large language models (MLLMs) for generating high-quality visual presentations. PreGenie is built on the Slidev presentation framework, where slides are rendered from Markdown code. It operates in two stages: (1) Analysis and Initial Generation, which summarizes multimodal input and generates initial code, and (2) Review and Re-generation, which iteratively reviews intermediate code and rendered slides to produce final, high-quality presentations. Each stage leverages multiple MLLMs that collaborate and share information. Comprehensive experiments demonstrate that PreGenie excels in multimodal understanding, outperforming existing models in both aesthetics and content consistency, while aligning more closely with human design preferences.

cs.LG↗

arXiv2025

Floquet Möbius topological insulators

Möbius topological insulators have dispersive edge bands with Möbius twists in momentum space, which are protected by the combination of chiral and $Z_2$-projective translational symmetries. In this work, we reveal a unique type of Möbius topological insulator, whose edge bands could twist around the quasienergy $π$ of a periodically driven system and are thus of Floquet origin. By applying time-periodic quenches to an experimentally realized Möbius insulator model, we obtain interconnected Möbius edge bands around zero and $π$ quasienergies, which can coexist with a gapped or gapless bulk. These Möbius bands are topologically characterized by a pair of generalized winding numbers, which are integer-quantized due to an emergent chiral symmetry at a high-symmetry point in momentum space. Numerical investigations of the quasienergy and entanglement spectra provide consistent evidence for the presence of such Möbius topological phases. A protocol based on the adiabatic switching of edge-band populations is further introduced to dynamically characterize the topology of Floquet Möbius edge bands. Our findings thus extend the scope of Möbius topological phases to nonequilibrium settings and unveil a unique class of Möbius-twisted topological edge states without static counterparts.

cond-mat.mes-hall↗

arXiv2025

Transcranial Photoacoustic Imaging for Human Intracranial Pressure Evaluation

Photoacoustic imaging (PAI), by combining high optical contrast with ultrasonic resolution, offers a promising noninvasive approach for dynamic monitoring of cerebral vasculature. However, transcranial PAI still faces significant challenges due to strong attenuation of both optical and acoustic signals by the skull. In this study, we propose a multi-wavelength photoacoustic tomography system and method for intracranial pressure (ICP) assessment, enabling visualization of cross-sectional structures of the middle cerebral artery (MCA) through the human temporal bone. By utilizing multi-wavelength excitation in the near-infrared-I (NIR-I) window, quantitative maps of blood oxygen saturation ($\mathbf{sO_2}$) are reconstructed, and the relationship between oxygenation dynamics and ICP variations is established. Experimental results demonstrate that the proposed system can successfully capture dynamic $\mathbf{sO_2}$ fluctuations in the MCA despite skull attenuation, revealing its characteristic responses to ICP changes. This work provides a high-precision, noninvasive imaging tool for early stroke diagnosis, cerebral vascular function assessment, and neurointerventional guidance, highlighting the clinical translational potential of PAI in neuroscience.

physics.med-ph↗

arXiv2025

Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models

Recent advancements in visual generative models have enabled high-quality image and video generation, opening diverse applications. However, evaluating these models often demands sampling hundreds or thousands of images or videos, making the process computationally expensive, especially for diffusion-based models with inherently slow sampling. Moreover, existing evaluation methods rely on rigid pipelines that overlook specific user needs and provide numerical results without clear explanations. In contrast, humans can quickly form impressions of a model's capabilities by observing only a few samples. To mimic this, we propose the Evaluation Agent framework, which employs human-like strategies for efficient, dynamic, multi-round evaluations using only a few samples per round, while offering detailed, user-tailored analyses. It offers four key advantages: 1) efficiency, 2) promptable evaluation tailored to diverse user needs, 3) explainability beyond single numerical scores, and 4) scalability across various models and tools. Experiments show that Evaluation Agent reduces evaluation time to 10% of traditional methods while delivering comparable results. The Evaluation Agent framework is fully open-sourced to advance research in visual generative models and their efficient evaluation.

cs.CV↗

arXiv2025

VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness

Video generation has advanced significantly, evolving from producing unrealistic outputs to generating videos that appear visually convincing and temporally coherent. To evaluate these video generative models, benchmarks such as VBench have been developed to assess their faithfulness, measuring factors like per-frame aesthetics, temporal consistency, and basic prompt adherence. However, these aspects mainly represent superficial faithfulness, which focus on whether the video appears visually convincing rather than whether it adheres to real-world principles. While recent models perform increasingly well on these metrics, they still struggle to generate videos that are not just visually plausible but fundamentally realistic. To achieve real "world models" through video generation, the next frontier lies in intrinsic faithfulness to ensure that generated videos adhere to physical laws, commonsense reasoning, anatomical correctness, and compositional integrity. Achieving this level of realism is essential for applications such as AI-assisted filmmaking and simulated world modeling. To bridge this gap, we introduce VBench-2.0, a next-generation benchmark designed to automatically evaluate video generative models for their intrinsic faithfulness. VBench-2.0 assesses five key dimensions: Human Fidelity, Controllability, Creativity, Physics, and Commonsense, each further broken down into fine-grained capabilities. Tailored to individual dimensions, our evaluation framework integrates generalists such as SOTA VLMs and LLMs, and specialists, including anomaly detection methods proposed for video generation. We conduct extensive human annotations to ensure evaluation alignment with human judgment. By pushing beyond superficial faithfulness toward intrinsic faithfulness, VBench-2.0 aims to set a new standard for the next generation of video generative models in pursuit of intrinsic faithfulness.

cs.CV↗

arXiv2025

AdaptiveAE: An Adaptive Exposure Strategy for HDR Capturing in Dynamic Scenes

Mainstream high dynamic range imaging techniques typically rely on fusing multiple images captured with different exposure setups (shutter speed and ISO). A good balance between shutter speed and ISO is crucial for achieving high-quality HDR, as high ISO values introduce significant noise, while long shutter speeds can lead to noticeable motion blur. However, existing methods often overlook the complex interaction between shutter speed and ISO and fail to account for motion blur effects in dynamic scenes. In this work, we propose AdaptiveAE, a reinforcement learning-based method that optimizes the selection of shutter speed and ISO combinations to maximize HDR reconstruction quality in dynamic environments. AdaptiveAE integrates an image synthesis pipeline that incorporates motion blur and noise simulation into our training procedure, leveraging semantic information and exposure histograms. It can adaptively select optimal ISO and shutter speed sequences based on a user-defined exposure time budget, and find a better exposure schedule than traditional solutions. Experimental results across multiple datasets demonstrate that it achieves the state-of-the-art performance.

cs.CV↗

arXiv2025

Little Red reionization factories

In this note, we suggest the possibility that an elucidation of the nature of the numerous Little Red Dots (LRDs) at high redshifts, may facilitate the resolution of another concurrent cosmic puzzle, namely reionization. Specifically, it is hypothesized that intergalactic HI gas is compelled by the tidal field associated with a growing gravitational entropy, in the form of gas streams, into colliding at the LRD sites. The resulting shock heating and compression encourage starburst activities, which subsequently photo-ionize the gas into HII, that in turn shines into the rest-optical via bremsstrahlung radiation within the minimal-timescale-cutoff (i.e., red) regime, before escaping back into the intergalactic space, propelled by the newly fusion-injected energy.

astro-ph.GA↗

arXiv2025

SpikeSTAG: Spatial-Temporal Forecasting via GNN-SNN Collaboration

Spiking neural networks (SNNs), inspired by the spiking behavior of biological neurons, offer a distinctive approach for capturing the complexities of temporal data. However, their potential for spatial modeling in multivariate time-series forecasting remains largely unexplored. To bridge this gap, we introduce a brand new SNN architecture, which is among the first to seamlessly integrate graph structural learning with spike-based temporal processing for multivariate time-series forecasting. Specifically, we first embed time features and an adaptive matrix, eliminating the need for predefined graph structures. We then further learn sequence features through the Observation (OBS) Block. Building upon this, our Multi-Scale Spike Aggregation (MSSA) hierarchically aggregates neighborhood information through spiking SAGE layers, enabling multi-hop feature extraction while eliminating the need for floating-point operations. Finally, we propose a Dual-Path Spike Fusion (DSF) Block to integrate spatial graph features and temporal dynamics via a spike-gated mechanism, combining LSTM-processed sequences with spiking self-attention outputs, effectively improve the model accuracy of long sequence datasets. Experiments show that our model surpasses the state-of-the-art SNN-based iSpikformer on all datasets and outperforms traditional temporal models at long horizons, thereby establishing a new paradigm for efficient spatial-temporal modeling.

cs.LG↗

arXiv2025

Ear-Keeper: A Cross-Platform AI System for Rapid and Accurate Ear Disease Diagnosis

Early and accurate detection systems for ear diseases, powered by deep learning, are essential for preventing hearing impairment and improving population health. However, the limited diversity of existing otoendoscopy datasets and the poor balance between diagnostic accuracy, computational efficiency, and model size have hindered the translation of artificial intelligence (AI) algorithms into healthcare applications. In this study, we constructed a large-scale, multi-center otoendoscopy dataset covering eight common ear diseases and healthy cases. Building upon this resource, we developed Best-EarNet, an ultrafast and lightweight deep learning architecture integrating a novel Local-Global Spatial Feature Fusion Module with a multi-scale supervision strategy, enabling real-time and accurate classification of ear conditions. Leveraging transfer learning, Best-EarNet, with a model size of only 2.94 MB, achieved diagnostic accuracies of 95.23% on an internal test set (22,581 images) and 92.14% on an external test set (1,652 images), while requiring only 0.0125 seconds (80 frames per second) to process a single image on a standard CPU. Further subgroup analysis by gender and age showed consistently excellent performance of Best-EarNet across all demographic groups. To enhance clinical interpretability and user trust, we incorporated Grad-CAM-based visualization, highlighting the specific abnormal ear regions contributing to AI predictions. Most importantly, we developed Ear-Keeper, a cross-platform intelligent diagnosis system built upon Best-EarNet, deployable on smartphones, tablets, and personal computers. Ear-Keeper enables public users and healthcare providers to perform comprehensive real-time video-based ear canal screening, supporting early detection and timely intervention of ear diseases.

cs.CV↗