arXiv subjects

Xiaoming Peng

Publications and source records attributed to Xiaoming Peng.

3 recordsLinked to original sources

arXiv2026

Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs

Large language models (LLM) have been the key focus of recent machine learning breakthroughs. As the model size grows exponentially, the computation needed for training is astronomical. A GPU can do a huge amount of arithmetic. However, the data its compute units need is not always available when they are ready. This data may still be in slow memory or on another GPU. While the chip waits, expensive compute sits idle, and at scale even a small drop in utilization translates into substantial cost. In this paper, we treat this waiting time as the primary bottleneck. We focus on hardware features designed for explicit data movement. These features load data from off-chip memory onto the chip, rearrange it into the layout the computation requires as it is loaded, and transfer it between GPUs. By orchestrating them, we keep a steady pipeline of data flowing to the compute units so that they remain busy. We present kernel and framework implementations that address this bottleneck using three key data-movement instructions and hardware features. The Tensor Data Mover (TDM) moves tensor data from high-bandwidth memory (HBM) into on-chip memory. Transpose-load instructions transpose tensor data into the layout that LLM computations require as it is loaded on-chip. Finally, System DMA (SDMA) and Remote DMA (RDMA) move data between GPUs over the scale-up and scale-out networks, respectively. Our results show that TDM and transpose-load provide on average 1.88x and 1.19x speedup at the kernel level, respectively. The SDMA and RDMA lead to on average 1.16x and 1.33x speedup for computation overlapped with communication. This is the first AMD LLM training system to utilize all the GPU data pipelining features. Our implementation has been integrated into open source framework and library for AMD GPUs. https://github.com/AMD-AGI/Primus-Turbo https://github.com/AMD-AGI/Primus

cs.AR↗

arXiv2024

Global existence and Rayleigh-Taylor instability for the semi-dissipative Boussinesq system with Naiver boundary conditions

Considered herein is the global existence of weak, strong solutions and Rayleigh-Taylor (RT) instability for 2D semi-dissipative Boussinesq equations in an infinite strip domain $Ω_{\infty}$ subject to Navier boundary conditions with non-positive slip coefficients. We first prove the global existence of weak and strong solutions on bounded domain $Ω_{R}$ via the Galerkin method, characteristic analyzing technique and Stokes estimates etc. Based on above results, we further derive the uniform estimates, independent of the length of horizontal direction of $Ω_{R}$, ensuring the global existence of weak and strong solutions in unbounded case $Ω_{\infty}$ by utilizing the domain expansion method. Moreover, when the steady temperature is higher with decreasing height (i.e., RT steady-state) on certain region, we demonstrate that the steady-state is linear unstable through the construction of energy functional and the settlement of a family of modified variational problems. Furthermore, with the help of unstable solutions constructed in linear instability and global existence theorems, we confirm the instability of nonlinear problem in a Lipschitz structural sense. Finally, we give a series of rigorous verification (see Appendix) including the spectra of Stokes equations with Navier boundary conditions, Sobolev embedding inequalities, trace inequalities, and Stokes estimates under Navier boundary conditions etc, used in the proof of main conclusions.

math.DS↗

arXiv2020

Massive MIMO for Cellular-Connected UAV: Challenges and Promising Solutions

Massive multiple-input multiple-output (MIMO) is a promising technology for enabling cellular-connected unmanned aerial vehicle (UAV) communications in the future. Equipped with full-dimensional large arrays, ground base stations (GBSs) can apply adaptive fine-grained three-dimensional (3D) beamforming to mitigate the strong interference between high-altitude UAVs and low-altitude terrestrial users, thus significantly enhancing the network spectral efficiency. However, the performance gain of massive MIMO critically depends on the accurate channel state information (CSI) of both UAVs and terrestrial users at the GBSs, which is practically difficult to achieve due to UAV-induced pilot contamination and UAV's high mobility in 3D. Moreover, the increasingly popular applications relying on a large group of coordinated UAVs or UAV swarm as well as the practical hybrid GBS beamforming architecture for massive MIMO further complicate the pilot contamination and channel/beam tracking problems. In this article, we provide an overview of the above challenging issues, propose new solutions to cope with them, and discuss about promising directions for future research. Preliminary simulation results are also provided to validate the effectiveness of proposed solutions.

cs.IT↗