Advanced 400G-SerDes Modulations and Equalizers for GPU Cluster in AI Era
High-speed GPU cluster interconnects are evolving from 200 to 400 Gbps to meet the growing computing demands of artificial intelligence. We investigate high-order modulation and advanced equalization for 400 Gbps interconnects. Theoretical bit error rate (BER) expressions for 4/8-ary pulse amplitude modulation (PAM) and approximate expressions for six PAM6 formats generated from 32-ary quadrature amplitude modulation (32QAM) are derived. In additive white Gaussian noise, PAM6 generated by framed-cross 32QAM achieves the best BER performance among the PAM6 formats. PAM6 and PAM8 incur peak signal-to-noise ratio (PSNR) penalties of approximately 4.2 and 7.2 dB, respectively, relative to PAM4 at the KP4 forward error correction (KP4-FEC) limit. Simulations compare PAM4, PAM6, and PAM8 at a line rate of 440 Gbps (net rate of 400G) under different bandwidth limitations. As bandwidth becomes more constrained, bandwidth requirements increasingly dominate performance over noise tolerance. PAM4 requires the lowest PSNR to reach the KP4-FEC limit in mildly bandwidth-limited channels, whereas PAM6 performs best under moderate limitations. PAM8 offers a significant advantage under severe limitations because of its lower bandwidth requirement and reduced susceptibility to high-frequency attenuation. With maximum likelihood sequence estimation, PAM6 and PAM8 reach the KP4-FEC limit under all evaluated channel conditions, while PAM6 requires a lower PSNR in most channels, highlighting its potential for 400G-SerDes systems.