arXiv subjects

Zhongxing Xu

Publications and source records attributed to Zhongxing Xu.

23 records · Page 2Linked to original sources

arXiv2024

OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining

Surgical practice involves complex visual interpretation, procedural skills, and advanced medical knowledge, making surgical vision-language pretraining (VLP) particularly challenging due to this complexity and the limited availability of annotated data. To address the gap, we propose OphCLIP, a hierarchical retrieval-augmented vision-language pretraining framework specifically designed for ophthalmic surgical workflow understanding. OphCLIP leverages the OphVL dataset we constructed, a large-scale and comprehensive collection of over 375K hierarchically structured video-text pairs with tens of thousands of different combinations of attributes (surgeries, phases/operations/actions, instruments, medications, as well as more advanced aspects like the causes of eye diseases, surgical objectives, and postoperative recovery recommendations, etc). These hierarchical video-text correspondences enable OphCLIP to learn both fine-grained and long-term visual representations by aligning short video clips with detailed narrative descriptions and full videos with structured titles, capturing intricate surgical details and high-level procedural insights, respectively. Our OphCLIP also designs a retrieval-augmented pretraining framework to leverage the underexplored large-scale silent surgical procedure videos, automatically retrieving semantically relevant content to enhance the representation learning of narrative videos. Evaluation across 11 datasets for phase recognition and multi-instrument identification shows OphCLIP's robust generalization and superior performance.

cs.CV↗

arXiv2024

OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

Surgical scene perception via videos is critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets face challenges such as small scale, lack of diversity in surgery and phase categories, and absence of time-localized annotations. These limitations impede action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 fine-grained operations. 2) Sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability. 3) Time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 285 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Code and dataset are available at: https://minghu0830.github.io/OphNet-benchmark/.

cs.CV↗

arXiv2024

Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation

Recent weakly supervised semantic segmentation (WSSS) methods strive to incorporate contextual knowledge to improve the completeness of class activation maps (CAM). In this work, we argue that the knowledge bias between instances and contexts affects the capability of the prototype to sufficiently understand instance semantics. Inspired by prototype learning theory, we propose leveraging prototype awareness to capture diverse and fine-grained feature attributes of instances. The hypothesis is that contextual prototypes might erroneously activate similar and frequently co-occurring object categories due to this knowledge bias. Therefore, we propose to enhance the prototype representation ability by mitigating the bias to better capture spatial coverage in semantic object regions. With this goal, we present a Context Prototype-Aware Learning (CPAL) strategy, which leverages semantic context to enrich instance comprehension. The core of this method is to accurately capture intra-class variations in object features through context-aware prototypes, facilitating the adaptation to the semantic attributes of various instances. We design feature distribution alignment to optimize prototype awareness, aligning instance feature distributions with dense features. In addition, a unified training framework is proposed to combine label-guided classification supervision and prototypes-guided self-supervision. Experimental results on PASCAL VOC 2012 and MS COCO 2014 show that CPAL significantly improves off-the-shelf methods and achieves state-of-the-art performance. The project is available at https://github.com/Barrett-python/CPAL.

cs.CV↗

arXiv2022

Multireference configuration interaction study of the predissociation of C$_{2}$ via its $F\,^1Π_u$ state

Photodissociation is one of the main destruction pathways for dicarbon (C$_{2}$) in astronomical environments such as diffuse interstellar clouds, yet the accuracy of modern astrochemical models is limited by a lack of accurate photodissociation cross sections in the vacuum ultraviolet range. C$_{2}$ features a strong predissociative $F\,^1Π_u - X\,^1Σ_g^+$ electronic transition near 130 nm originally measured in 1969; however, no experimental studies of this transition have been carried out since, and theoretical studies of the $F\,^1Π_u$ state are limited. In this work, potential energy curves of excited electronic states of C$_{2}$ are calculated with the aim of describing the predissociative nature of the $F\,^1Π_u$ state and providing new ab initio photodissociation cross sections for astrochemical applications. Accurate electronic calculations of 56 singlet, triplet, and quintet states are carried out at the DW-SA-CASSCF/MRCI+Q level of theory with a CAS(8,12) active space and the aug-cc-pV5Z basis set augmented with additional diffuse functions. Photodissociation cross sections arising from the vibronic ground state to the $F\,^1Π_u$ state are calculated by a coupled-channel model. The total integrated cross section through the $F\,^1Π_u$ $v=0$ and $v=1$ bands is 1.198$\times$10$^{-13} $cm$^2$cm$^{-1}$, giving rise to a photodissociation rate of 5.02$\times$10$^{-10}$ s$^{-1}$ under the standard interstellar radiation field, much larger than the rate in the Leiden photodissociation database. In addition, we report a new $2\,^1Σ_u^+$ state that should be detectable via a strong $2\,^1Σ_u^+-X\,^1Σ_g^+$ band around 116 nm.

astro-ph.GA↗

arXiv2019

Ab initio Study of Ground-State CS Photodissociation Via Highly Excited Electronic States

Photodissociation by ultraviolet radiation is the key destruction pathway for CS in photon-dominated regions, such as diffuse clouds. However, the large uncertainties of photodissociation cross sections and rates of CS, resulting from a lack of both laboratory experiments and theoretical calculations, limit the accuracy of calculated abundances of S-bearing molecules by modern astrochemical models. Here we show a detailed \textit{ab initio} study of CS photodissociation. Accurate potential energy curves of CS electronic states were obtained by choosing an active space CAS(8,10) in MRCI+Q/aug-cc-pV(5+d)Z calculation with additional diffuse functions, with a focus on the $B$ and $C\,^1Σ^+$ states. Cross sections for both direct photodissociation and predissociation from the vibronic ground state were calculated by applying the coupled-channel method. We found that the $C-X$ $(0-0)$ transition has extremely strong absorption due to a large transition dipole moment in the Franck-Condon region and the upper state is resonant with several triplet states via spin-orbit couplings, resulting in predissociation to the main atomic products C $(^3P)$ and S $(^1D)$. Our new calculations show the photodissociation rate under the standard interstellar radiation field is $2.9\ee{-9}$\,s$^{-1}$, with a 57\% contribution from $C-X$ $(0-0)$ transition. This value is larger than that adopted by the Leiden photodissociation and photoionization database by a factor of 3.0. Our accurate \textit{ab initio} calculations will allow more secure determination of S-bearing molecules in astrochemical models.

astro-ph.GA↗