CSF-Net: Cross-Modal Spatiotemporal Fusion Network for Pulmonary Nodule Malignancy Predicting

Abstract

Pulmonary nodules are an early sign of lung cancer, and detecting them early is vital for improving patient survival rates. Most current methods use only single Computed Tomography (CT) images to assess nodule malignancy. However, doctors typically make a comprehensive assessment in clinical practice by integrating follow-up CT scans with clinical data. To enhance this process, our study introduces a Cross-Modal Spatiotemporal Fusion Network, named CSF-Net, designed to predict the malignancy of pulmonary nodules using follow-up CT scans. This approach simulates the decision-making process of clinicians who combine follow-up imaging with clinical information. CSF-Net comprises three key components: spatial feature extraction module, temporal residual fusion module, and cross-modal attention fusion module. Together, these modules enable precise predictions of nodule malignancy. Additionally, we utilized the publicly available NLST dataset to screen and annotate the specific locations of pulmonary nodules and created a new dataset named NLST-cmst. Our experimental results on the NLST-cmst dataset demonstrate significant performance improvements, with an accuracy of 0.8974, a precision of 0.8235, an F1 score of 0.8750, an AUC of 0.9389, and a recall of 0.9333. These findings indicate that our multimodal spatiotemporal fusion approach, which combines follow-up data with clinical information, surpasses existing methods, underscoring its effectiveness in predicting nodule malignancy.