arXiv subjects

Jiaqi Fan

Publications and source records attributed to Jiaqi Fan.

16 recordsLinked to original sources

arXiv2026

CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy

Automatic neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved encouraging results on local image blocks, it remains unclear whether such progress translates to reconstruction that is both structurally accurate and scalable to the whole-brain scale. We present CORAL, the first benchmark for structure-aware evaluation of automatic neuron reconstruction from light microscopy images at both local and whole-brain scales. Built on a high-quality whole-brain fMOST dataset with carefully curated annotations, CORAL establishes two progressive tasks: block-level reconstruction, which evaluates reconstruction methods under limited spatial context, and brain-wide reconstruction, which assesses complete neuron reconstruction at the whole-brain scale. To account for topological correctness beyond geometric distance similarity, we introduce a structure-aware metric based on fiber prediction. To further achieve complete neuron reconstruction across the entire brain, we develop a brain-wide neuron tracing framework that extends arbitrary local reconstruction methods to the whole-brain scale through an iterative local-to-global process. Using this benchmark, we provide the first structure-aware comparison of mainstream methods for local neuron reconstruction and further evaluate their performance in brain-wide reconstruction. Our results underscore the importance of structure-aware evaluation and the need for more robust methods for complete neuron reconstruction.

cs.CV↗

arXiv2026

Localization Lengths of Power-Law Random Band Matrices

We study large $N\times N$ power-law random band matrices $H=(H_{ij})$ with centered complex Gaussian entries, where the variances satisfy a power-law decay $\mathbb{E}|H_{ij}|^2\propto (|i-j|/W+1)^{-1-\alpha}$, for some exponent $\alpha>-1$ and bandwidth $W\gg 1$. We establish the following lower bounds, with high probability, on the localization length $\ell$ of bulk eigenvectors in the different regimes of $\alpha$: (1) $\ell=N$ if $-1<\alpha<0$; (2) $\ell \ge W^{C}$ for any large constant $C>0$ if $0 < \alpha <1$; (3) $\ell \ge W^{\alpha/(\alpha-1)}$ if $1 < \alpha <2$; (4) $\ell \ge W^{2}$ if $ \alpha > 2$. These results verify the physical conjecture of arXiv:cond-mat/9604163 on the delocalized side. The main difficulty in the proof lies in handling the interplay between the non-mean-field nature of the model and the slow decay of the variance profile. To address this issue, a key technical ingredient is a new dynamical analysis of $T$-variables formed from pairs of resolvent entries of $H$. In contrast to the fundamental works on regular random band matrices with fast-decaying variances in arXiv:2501.01718 and arXiv:2506.06441, this approach does not rely on higher-order resolvent loops.

math.PR↗

arXiv2026

RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models

Reinforcement learning (RL) shows promise for enhancing LLM agentic reasoning, yet sparse terminal rewards hinder fine-grained optimization. Process reward modeling offers an alternative but incurs high computational costs, reward hacking risks, and annotation bottlenecks. We introduce RewardFlow, a lightweight method for estimating state-level rewards in agentic reasoning. By constructing state graphs that capture the intrinsic topological structure of trajectories, RewardFlow performs topology-aware propagation to estimate each state's contribution to success, yielding principled, annotation-free dense rewards. Used for RL optimization, RewardFlow substantially outperforms prior baselines across four agentic benchmarks: +6.2% average success rate on text-based tasks, +29.7% on visual reasoning over the strongest baseline across three model scales, and +10% accuracy on DeepResearch, with superior robustness and training efficiency. The implementation of RewardFlow is publicly available at https://github.com/tmlr-group/RewardFlow.

cs.AI↗

arXiv2025

A Block Reduction Method for Random Band Matrices with General Variance Profiles

We present a novel block reduction method for the study of a general class of random band matrices (RBM) defined on the $d$-dimensional lattice $\mathbb{Z}_{L}^d:=\{1,2,\ldots,L\}^{d}$ for $d\in \{1,2\}$, with band width $W$ and an almost arbitrary variance profile subject to a core condition. We prove the delocalization of bulk eigenvectors for such RBMs under the assumptions $W\ge L^{1/2+\varepsilon}$ in one dimension and $W\geq L^{\varepsilon}$ in two dimensions, where $\varepsilon$ is an arbitrarily small constant. This result extends the findings of arXiv:2501.01718 and arXiv:2503.07606 on block RBMs to models with general variance profiles. Furthermore, we generalize our results to Wegner orbital models with small interaction strength $\lambda\ll 1$. Under the sharp condition $\lambda\gg W^{-d/2}$, we establish optimal lower bounds for the localization lengths of bulk eigenvectors, thereby extending the results of arXiv:2503.11382 to settings with nearly arbitrary potential and hopping terms. Our block reduction method provides a powerful and flexible framework that reduces both the dynamical analysis of the loop hierarchy and the derivation of deterministic estimates for general RBMs to the corresponding analysis of block RBMs, as developed in arXiv:2501.01718, arXiv:2503.07606 and arXiv:2503.11382.

math.PR↗

arXiv2025

Localization-delocalization transition for a random block matrix model at the edge

Consider a random block matrix model consisting of $D$ random systems arranged along a circle, where each system is modeled by an independent $N\times N$ complex Hermitian Wigner matrix. Neighboring systems interact via an arbitrary deterministic $N\times N$ matrix $A$. In this paper, we extend the localization-delocalization transition previously established in arxiv:2312.07297 for the bulk eigenvalue spectrum to the entire spectrum, including the spectral edges. Let $[E^-,E^+]$ denote the support of the limiting spectral density, and define $\kappa_E:=|E-E^+|\wedge |E-E^-|$ as the distance from a given energy $E \in [E^-, E^+]$ to the spectral edges. We show that for eigenvalues near $E$, the corresponding eigenvectors undergo a localization-delocalization transition when $\|A\|_{\mathrm{HS}}$ crosses the critical threshold $(\kappa_E + N^{-2/3})^{-1/2}$. In the delocalized phase, the extreme eigenvalues asymptotically follow the Tracy-Widom distribution, while in the localized phase, the edge eigenvalue statistics asymptotically match those of $D$ independent GUE ensembles, up to a deterministic shift. Our results recover those of arxiv:2312.07297 in the bulk regime, where $\kappa_E \asymp 1$, and further reveal the presence of mobility edges near $E^\pm$ when $1 \ll \|A\|_{\mathrm{HS}} \ll N^{1/3}$. Specifically, bulk eigenvectors corresponding to energies $E$ with $\kappa_E \gg \|A\|_{\mathrm{HS}}^{-2}$ are delocalized, while those with $\kappa_E \ll \|A\|_{\mathrm{HS}}^{-2}$ are localized.

math.PR↗

arXiv2024

MLLM-SUL: Multimodal Large Language Model for Semantic Scene Understanding and Localization in Traffic Scenarios

Multimodal large language models (MLLMs) have shown satisfactory effects in many autonomous driving tasks. In this paper, MLLMs are utilized to solve joint semantic scene understanding and risk localization tasks, while only relying on front-view images. In the proposed MLLM-SUL framework, a dual-branch visual encoder is first designed to extract features from two resolutions, and rich visual information is conducive to the language model describing risk objects of different sizes accurately. Then for the language generation, LLaMA model is fine-tuned to predict scene descriptions, containing the type of driving scenario, actions of risk objects, and driving intentions and suggestions of ego-vehicle. Ultimately, a transformer-based network incorporating a regression token is trained to locate the risk objects. Extensive experiments on the existing DRAMA-ROLISP dataset and the extended DRAMA-SRIS dataset demonstrate that our method is efficient, surpassing many state-of-the-art image-based and video-based methods. Specifically, our method achieves 80.1% BLEU-1 score and 298.5% CIDEr score in the scene understanding task, and 59.6% accuracy in the localization task. Codes and datasets are available at https://github.com/fjq-tongji/MLLM-SUL.

cs.CV↗

arXiv2024

RAC3: Retrieval-Augmented Corner Case Comprehension for Autonomous Driving with Vision-Language Models

Understanding and addressing corner cases is essential for ensuring the safety and reliability of autonomous driving systems. Vision-language models (VLMs) play a crucial role in enhancing scenario comprehension, yet they face significant challenges, such as hallucination and insufficient real-world grounding, which compromise their performance in critical driving scenarios. In this work, RAC3, a novel framework designed to enhance the performance of VLMs in corner case comprehension, is proposed. RAC3 integrates a frequency-spatial fusion (FSF) image encoder, a cross-modal alignment training method for embedding models with hard and semi-hard negative mining, and a fast querying and retrieval pipeline based on K-Means clustering and hierarchical navigable small world (HNSW) indexing. A multimodal chain-of-thought (CoT) prompting strategy to guide analogical reasoning and reduce hallucinations during inference is introduced. Moreover, an update mechanism is integrated into RAC3 to ensure continual learning within the framework. Extensive experiments on the CODA and nuScenes datasets demonstrate that RAC3 significantly improves corner case comprehension across multiple downstream tasks. Compared to prior state-of-the-art methods, RAC3 achieves the highest final score of 74.46 on the CODA-LM benchmark and shows consistent performance gains when integrated with end-to-end frameworks like DriveLM. These results demonstrate the effectiveness of retrieval-augmented strategies and cross-modal alignment for safer and more interpretable autonomous driving.

cs.CV↗

arXiv2024

Hallucination Elimination and Semantic Enhancement Framework for Vision-Language Models in Traffic Scenarios

Large vision-language models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding and generation tasks. However, these models occasionally generate hallucinatory texts, resulting in descriptions that seem reasonable but do not correspond to the image. This phenomenon can lead to wrong driving decisions of the autonomous driving system. To address this challenge, this paper proposes HCOENet, a plug-and-play chain-of-thought correction method designed to eliminate object hallucinations and generate enhanced descriptions for critical objects overlooked in the initial response. Specifically, HCOENet employs a cross-checking mechanism to filter entities and directly extracts critical objects from the given image, enriching the descriptive text. Experimental results on the POPE benchmark demonstrate that HCOENet improves the F1-score of the Mini-InternVL-4B and mPLUG-Owl3 models by 12.58% and 4.28%, respectively. Additionally, qualitative results using images collected in open campus scene further highlight the practical applicability of the proposed method. Compared with the GPT-4o model, HCOENet achieves comparable descriptive performance while significantly reducing costs. Finally, two novel semantic understanding datasets, CODA_desc and nuScenes_desc, are created for traffic scenarios to support future research. The codes and datasets are publicly available at https://github.com/fjq-tongji/HCOENet.

cs.CV↗

arXiv2023

Understanding the Multi-modal Prompts of the Pre-trained Vision-Language Model

Prompt learning has emerged as an efficient alternative for fine-tuning foundational models, such as CLIP, for various downstream tasks. However, there is no work that provides a comprehensive explanation for the working mechanism of the multi-modal prompts. In this paper, we conduct a direct analysis of the multi-modal prompts by asking the following questions: $(i)$ How do the learned multi-modal prompts improve the recognition performance? $(ii)$ What do the multi-modal prompts learn? To answer these questions, we begin by isolating the component of the formula where the prompt influences the calculation of self-attention at each layer in two distinct ways, \ie, $(1)$ introducing prompt embeddings makes the $[cls]$ token focus on foreground objects. $(2)$ the prompts learn a bias term during the update of token embeddings, allowing the model to adapt to the target domain. Subsequently, we conduct extensive visualization and statistical experiments on the eleven diverse downstream recognition datasets. From the experiments, we reveal that the learned prompts improve the performance mainly through the second way, which acts as the dataset bias to improve the recognition performance of the pre-trained model on the corresponding dataset. Meanwhile, we propose the bias tuning way to validate our finding. With a deeper understanding of the multi-modal prompt, we hope our work can inspire new and solid research in this direction.

cs.CV↗

arXiv2023

SKDF: A Simple Knowledge Distillation Framework for Distilling Open-Vocabulary Knowledge to Open-world Object Detector

In this paper, we attempt to specialize the VLM model for OWOD tasks by distilling its open-world knowledge into a language-agnostic detector. Surprisingly, we observe that the combination of a simple \textbf{knowledge distillation} approach and the automatic pseudo-labeling mechanism in OWOD can achieve better performance for unknown object detection, even with a small amount of data. Unfortunately, knowledge distillation for unknown objects severely affects the learning of detectors with conventional structures for known objects, leading to catastrophic forgetting. To alleviate these problems, we propose the \textbf{down-weight loss function} for knowledge distillation from vision-language to single vision modality. Meanwhile, we propose the \textbf{cascade decouple decoding structure} that decouples the learning of localization and recognition to reduce the impact of category interactions of known and unknown objects on the localization learning process. Ablation experiments demonstrate that both of them are effective in mitigating the impact of open-world knowledge distillation on the learning of known objects. Additionally, to alleviate the current lack of comprehensive benchmarks for evaluating the ability of the open-world detector to detect unknown objects in the open world, we propose two benchmarks, which we name "\textbf{StandardSet}$\heartsuit$" and "\textbf{IntensiveSet}$\spadesuit$" respectively, based on the complexity of their testing scenarios. Comprehensive experiments performed on OWOD, MS-COCO, and our proposed benchmarks demonstrate the effectiveness of our methods. The code and proposed dataset are available at \url{https://github.com/xiaomabufei/SKDF}.

cs.CV↗

arXiv2023

Detecting the open-world objects with the help of the Brain

Open World Object Detection (OWOD) is a novel computer vision task with a considerable challenge, bridging the gap between classic object detection (OD) benchmarks and real-world object detection. In addition to detecting and classifying seen/known objects, OWOD algorithms are expected to detect unseen/unknown objects and incrementally learn them. The natural instinct of humans to identify unknown objects in their environments mainly depends on their brains' knowledge base. It is difficult for a model to do this only by learning from the annotation of several tiny datasets. The large pre-trained grounded language-image models - VL (\ie GLIP) have rich knowledge about the open world but are limited to the text prompt. We propose leveraging the VL as the ``Brain'' of the open-world detector by simply generating unknown labels. Leveraging it is non-trivial because the unknown labels impair the model's learning of known objects. In this paper, we alleviate these problems by proposing the down-weight loss function and decoupled detection structure. Moreover, our detector leverages the ``Brain'' to learn novel objects beyond VL through our pseudo-labeling scheme.

cs.CV↗

arXiv2023

Predictions of photophysical properties of phosphorescent platinum(II) complexes based on ensemble machine learning approach

Phosphorescent metal complexes have been under intense investigations as emissive dopants for energy efficient organic light emitting diodes (OLEDs). Among them, cyclometalated Pt(II) complexes are widespread triplet emitters with color-tunable emissions. To render their practical applications as OLED emitters, it is in great need to develop Pt(II) complexes with high radiative decay rate constant ($k_r$) and photoluminescence (PL) quantum yield. Thus, an efficient and accurate prediction tool is highly desirable. Here, we develop a general protocol for accurate predictions of emission wavelength, radiative decay rate constant, and PL quantum yield for phosphorescent Pt(II) emitters based on the combination of first-principles quantum mechanical method, machine learning (ML) and experimental calibration. A new dataset concerning phosphorescent Pt(II) emitters is constructed, with more than two hundred samples collected from the literature. Features containing pertinent electronic properties of the complexes are chosen. Our results demonstrate that ensemble learning models combined with stacking-based approaches exhibit the best performance, where the values of squared correlation coefficients ($R^2$), mean absolute error (MAE), and root mean square error (RMSE) are 0.96, 7.21 nm and 13.00 nm for emission wavelength prediction, and 0.81, 0.11 and 0.15 for PL quantum yield prediction. For radiative decay rate constant ($k_r$), the obtained value of $R^2$ is 0.67 while MAE and RMSE are 0.21 and 0.25 (both in log scale), respectively. The accuracy of the protocol is further confirmed using 24 recently reported Pt(II) complexes, which demonstrates its reliability for a broad palette of Pt(II) emitters.We expect this protocol will become a valuable tool, accelerating the rational design of novel OLED materials with desired properties.

cs.LG↗

arXiv2023

CAT: LoCalization and IdentificAtion Cascade Detection Transformer for Open-World Object Detection

Open-world object detection (OWOD), as a more general and challenging goal, requires the model trained from data on known objects to detect both known and unknown objects and incrementally learn to identify these unknown objects. The existing works which employ standard detection framework and fixed pseudo-labelling mechanism (PLM) have the following problems: (i) The inclusion of detecting unknown objects substantially reduces the model's ability to detect known ones. (ii) The PLM does not adequately utilize the priori knowledge of inputs. (iii) The fixed selection manner of PLM cannot guarantee that the model is trained in the right direction. We observe that humans subconsciously prefer to focus on all foreground objects and then identify each one in detail, rather than localize and identify a single object simultaneously, for alleviating the confusion. This motivates us to propose a novel solution called CAT: LoCalization and IdentificAtion Cascade Detection Transformer which decouples the detection process via the shared decoder in the cascade decoding way. In the meanwhile, we propose the self-adaptive pseudo-labelling mechanism which combines the model-driven with input-driven PLM and self-adaptively generates robust pseudo-labels for unknown objects, significantly improving the ability of CAT to retrieve unknown objects. Comprehensive experiments on two benchmark datasets, i.e., MS-COCO and PASCAL VOC, show that our model outperforms the state-of-the-art in terms of all metrics in the task of OWOD, incremental object detection (IOD) and open-set detection.

cs.CV↗

arXiv2021

Interpretable Compositional Convolutional Neural Networks

The reasonable definition of semantic interpretability presents the core challenge in explainable AI. This paper proposes a method to modify a traditional convolutional neural network (CNN) into an interpretable compositional CNN, in order to learn filters that encode meaningful visual patterns in intermediate convolutional layers. In a compositional CNN, each filter is supposed to consistently represent a specific compositional object part or image region with a clear meaning. The compositional CNN learns from image labels for classification without any annotations of parts or regions for supervision. Our method can be broadly applied to different types of CNNs. Experiments have demonstrated the effectiveness of our method.

cs.CV↗

arXiv2020

Efficient Unpaired Image Dehazing with Cyclic Perceptual-Depth Supervision

Image dehazing without paired haze-free images is of immense importance, as acquiring paired images often entails significant cost. However, we observe that previous unpaired image dehazing approaches tend to suffer from performance degradation near depth borders, where depth tends to vary abruptly. Hence, we propose to anneal the depth border degradation in unpaired image dehazing with cyclic perceptual-depth supervision. Coupled with the dual-path feature re-using backbones of the generators and discriminators, our model achieves $\mathbf{20.36}$ Peak Signal-to-Noise Ratio (PSNR) on NYU Depth V2 dataset, significantly outperforming its predecessors with reduced Floating Point Operations (FLOPs).

eess.IV↗

arXiv2020

On continuous images of self-similar sets

Let $(\mathcal{M}, c_k, n_k,\kappa)$ be a class of homogeneous Moran sets. Suppose $f(x,y)\in C^3$ is a function defined on $\mathbb{R}^2$. Given $E_1, E_2\in(\mathcal{M}, c_k, n_k,\kappa) $, in this paper, we prove, under some checkable conditions on the partial derivatives of $f(x,y)$, that $$f(E_1,E_2)=\{f(x,y):x\in E_1,y\in E_2\}$$ is exactly a closed interval or a union of finitely many closed intervals. Similar results for the homogeneous self-similar sets with arbitrary overlaps can be obtained. Further generalization is available for some inhomogeneous self-similar sets if we utilize the approximation theorem.

math.MG↗