arXiv subjects

Vinkle Srivastav

Publications and source records attributed to Vinkle Srivastav.

3 recordsLinked to original sources

arXiv2026

SurgTEMP: Temporal-Aware Surgical Video Question Answering with Text-guided Visual Memory for Laparoscopic Cholecystectomy

Surgical procedures are inherently complex and risky, requiring extensive expertise and constant focus to navigate evolving intraoperative scenes. Computer-assisted systems such as surgical visual question answering (VQA) offer promises for education and intraoperative support. Current surgical VQA research largely focuses on static frame analysis, overlooking rich temporal semantics. Surgical video question answering is further challenged by low visual contrast, its highly knowledge-driven nature, diverse analytical needs spanning scattered temporal windows, and the hierarchy from basic perception to high-level intraoperative assessment. To address these challenges, we propose SurgTEMP, a multimodal LLM framework featuring (i) a query-guided token selection module that builds hierarchical visual memory (spatial and temporal memory banks) and (ii) a Surgical Competency Progression (SCP) training scheme. Together, they enable effective modeling of variable-length surgical videos while preserving procedure-relevant cues and temporal coherence, and better support diverse downstream assessment tasks. To support model development, we introduce CholeVidQA-32K, a surgical video question answering dataset comprising 32K open-ended QA pairs and 3,855 video segments (approximately 128 h total) from laparoscopic cholecystectomy. The dataset is organized into a three-level hierarchy -- Perception, Assessment, and Reasoning -- spanning 11 tasks from instrument/action/anatomy perception to Critical View of Safety (CVS), intraoperative difficulty, skill proficiency, and adverse event assessment. In comprehensive evaluations against state-of-the-art open-source multimodal and video LLMs (fine-tuned and zero-shot), SurgTEMP achieves substantial performance improvements, advancing the state of video-based surgical VQA. The project page is available at: https://camma-public.github.io/SurgTEMP/

cs.CV↗

arXiv2026

Self-Supervised Multi-View 3D Gaze Target Estimation via Probabilistic Ray Marching

We present a self-supervised approach, Self-MVGTE, for estimating 3D gaze targets from multiple camera views. Unlike existing methods that independently estimate 2D gaze targets per camera view, Self-MVGTE predicts gaze targets directly in 3D space for the first time. Moreover, it does not require any ground-truth annotations from the target scene and uses only the multi-view input images from a calibrated camera setup, pseudo 2D gaze target labels from a monocular gaze target estimation model, and 3D gaze vectors from a monocular 3D gaze estimation model. A key challenge is that these pseudo labels are inherently noisy and multi-view inconsistent. To address this, we propose a probabilistic ray marching framework, which models the uncertainty of these pseudo labels and exploits 3D gaze vectors as geometric priors. Specifically, these gaze vectors are first integrated into the monocular gaze target estimation model to improve its generalization to unseen scenes, producing higher-quality pseudo labels. Then, for 3D gaze target estimation, we construct a 3D gaze cone by casting a bundle of rays from the eye position around the gaze vector to strictly constrain the solution space. Within this cone, we propose a depth-guided feature sampling strategy using off-the-shelf DINOv2 and Depth-Anything-3 models, and estimate a spatial likelihood distribution of the gaze target. Finally, we convert the pseudo gaze target labels into a target distribution and softly optimize the network. Extensive experiments on the MVGT dataset show that Self-MVGTE achieves state-of-the-art performance, surpassing existing fully-supervised baselines.

cs.CV↗

arXiv2026

SGRNet: Spatially Guided Radiology Network for Structured Radiological Reporting of Head and Neck Cancer

Automated radiological report generation can alleviate clinical workloads and eliminate observer variability. However, standard free-text generation models pose hallucination risks in dense regions and fail under data scarcity. We address these challenges in Head and Neck Cancer (HNC) from contrast-enhanced CT (CECT) imaging. To enforce factual safety, we reformulate report generation as an anatomically grounded, multi-label, structured reporting task, predicting localized tumor involvement across a hierarchical clinical schema. To bridge the visual gap from missing metabolic imaging (e.g., PET), we introduce SGRNet (Spatially Guided Radiology Network), incorporating two low-cost spatial priors: automated organ segmentations and weakly supervised tumor localization maps modeled via 3D Gaussian heatmaps. These priors are dynamically integrated via spatial feature modulation to guide the network toward subtle tumor-induced structural alterations. Evaluated on a multi-centric dataset of 184 paired HNC CECT volumes and reports, on five clinically salient, densely packed anatomical subsites, SGRNet achieves a mean Average Precision (mAP) of 0.60, an 8.8 percentage-point absolute improvement over strong volume-only 3D baselines.

cs.CV↗