Synesthesia of Machines (SoM)-Aided LiDAR Point Cloud Transmission for Collaborative Perception

Abstract

Collaborative perception enables more accurate and comprehensive scene understanding by learning how to share information between agents, with LiDAR point clouds providing essential precise spatial data. Due to the substantial data volume generated by LiDAR sensors, efficient point cloud transmission is essential for low-latency multi-agent collaboration. In this work, we propose an efficient, robust and applicable LiDAR point cloud transmission system via the Synesthesia of Machines (SoM), termed LiDAR Point Cloud Feature Transmission (LPC-FT), to support collaborative perception among multiple agents. Specifically, we employ a density-preserving deep point cloud compression method that encodes the complete point cloud into a downsampled efficient representation. To mitigate the effects of the wireless channel, we design a channel encoder module based on self-attention to enhance LiDAR point cloud features and a feature fusion module based on cross-attention to integrate features from transceivers. Furthermore, we utilize the nonlinear activation layer and transfer learning to improve the training of deep neural networks in the presence the digital channel noise. Experimental results demonstrate that the proposed LPC-FT is more robust and effective than traditional octree-based compression followed by channel coding, and outperforms state-of-the-art deep learning-based compression techniques and existing semantic communication methods, reducing the Chamfer Distance by 30% and improving the PSNR by 1.9 dB on average. Owing to its superior reconstruction performance and robustness against channel variations, LPC-FT is expected to support collaborative perception tasks.