Min Jeong Cho | arXiv Science

arXiv2026

A Specialized Large Multimodal Model for Interpreting PET/CT in Head and Neck Cancer

Background: Diagnosing head and neck cancer using PET/CT is clinically challenging and time-consuming due to the anatomical complexity of the region, motivating computer-aided diagnosis (CAD). Generalist Large Multimodal Models (LMMs) remain limited in medical contexts by insufficient domain-specific knowledge, privacy and security concerns, and verbosity, motivating specialized standalone LMMs. Purpose: We evaluated the feasibility of a specialized LMM for automated PET/CT interpretation in head and neck cancer using a large-scale multi-institutional PET/CT dataset, a tailored training curriculum, and autoregressive training. Methods: LLaVA-NeXT was fine-tuned using a two-level curriculum with image-conversation pairs curated by two radiologists from public data. The dataset included clinically important annotations such as primary tumor presence and metastatic lymph node location. Level 1 used 28,000 image-conversation pairs to learn basic information, including modality type and hypermetabolism. Level 2 used 12,975 pairs to learn primary tumor presence and the existence and anatomical location of cervical lymph node metastases. External validation included four institutions with diverse imaging devices. Results: The specialized LMM substantially outperformed ChatGPT and LLaVA-NeXT. In Level-2 external validation, ROUGE-L, ROUGE-S, Cosine Similarity, Precision, Recall, and F1 were 0.8751, 0.8794, 0.8324, 0.8794, 0.8711, and 0.8751, while generalist models consistently scored below 0.1. Primary tumor classification accuracy was 83.14 +/- 1.15% internally and 69.03 +/- 0.81% externally. For lymph node localization, the corresponding scores were 0.6389, 0.6257, 0.5287, 0.5782, 0.6371, and 0.6648. Conclusion: Specialized LMMs show promising results for fast, accurate PET/CT-based diagnostic support and medical education, highlighting their potential for clinical translation.

cs.CV↗

arXiv2025

GPDM: Generation-Prior Diffusion Model for Accelerated Direct Attenuation and Scatter Correction of Whole-body 18F-FDG PET

Accurate attenuation and scatter corrections are crucial in positron emission tomography (PET) imaging for accurate visual interpretation and quantitative analysis. Traditional methods relying on computed tomography (CT) or magnetic resonance imaging (MRI) have limitations in accuracy, radiation exposure, and applicability. Deep neural networks provide potential approaches to estimating attenuation and scatter-corrected (ASC) PET from non-attenuation and non-scatter-corrected (NASC) PET images based on VAE or CycleGAN. However, the limitations inherent to conventional GAN-based methods, such as unstable training and mode collapse, need further advancements. To address these limitations and achieve more accurate attenuation and scatter corrections, we propose a novel framework for generating high-quality ASC PET images from NASC PET images: Generation-Prior Diffusion Model (GPDM). Our GPDM framework is based on the Denoising Diffusion Probabilistic Model (DDPM), but instead of starting sampling from an entirely different image distribution, it begins from a distribution similar to the target images we aim to generate. This similar distribution is referred to as the Generation-Prior. By leveraging this Generation-Prior, the GPDM framework effectively reduces the number of sampling steps and generates more refined ASC PET images. Our experimental results demonstrate that GPDM outperforms existing methods in generating ASC PET images, achieving superior accuracy while significantly reducing sampling time. These findings highlight the potential of GPDM to address the limitations of conventional methods and establish a new standard for efficient and accurate attenuation and scatter correction in PET imaging.

physics.med-ph↗