arXiv subjects

Minchul Kang

Publications and source records attributed to Minchul Kang.

4 recordsLinked to original sources

arXiv2026

Parallelism Strategy Chaining for Fast Training Convergence

Selecting a parallelism strategy - the configuration of data, tensor, and pipeline parallelism degrees together with micro- and global-batch sizes - largely determines the training efficiency of large language models. State-of-the-art methods search for a parallelism strategy offline and select the single strategy that minimizes per-iteration time. But we find that they neglect the target validation perplexity and time-to-perplexity (TTP). In particular, our analysis reveals that the best strategy yielding the fastest perplexity improvement changes multiple times during training. As a result, state-of-the-art methods are 1.8-11.4x slower in TTP than the strategy sequence that selects the best strategy at each iteration. This paper proposes CONA, a new training method that introduces online strategy chaining. Instead of a single strategy selected offline, CONA ranks candidate strategies during training using a surrogate metric built from compute throughput and gradient statistics, and switches the current strategy to a new strategy with a higher metric. In our evaluation with GPT-3 1.3B, BERT-Large, and Llama-3.2-1B, CONA reaches the target validation perplexity 1.4-9.6x faster than state-of-the-art methods. Moreover, CONA closely tracks the perplexity achieved by the sequence that selects the best strategy at each iteration, within 2.6%.

cs.LG↗

arXiv2026

Unified KV Pooling to Accelerate Long-Context LLM Serving

Long-context LLM serving requires offloading KV caches to host-memory and SSDs, but existing mechanisms are not designed for such long contexts. We observe significant inefficiencies in current KV caching in long contexts: high serving latency ~30.7 s, exceeding the typical TTFT requirement of 10 s by more than 3x. Our in-depth analysis explains two major reasons: (1) retrieval is serialized through host-memory and SSD, leaving other host-memory modules and SSDs underutilized, and (2) SSD-based KV retrieval spends 84% of its time in the kernel filesystem rather than actual device access. To address the problems, we propose unified KV pooling, which aggregates multiple host-memory modules and SSDs into a single logical pool and distributes KV caches across devices based on their bandwidth. To eliminate the filesystem overhead, we design KV-passthrough, which bypasses the kernel filesystem and directly accesses SSD-resident KV caches from user space via SPDK. Across evaluations on LLaMA 3.1-8B, GPT-OSS-20B, and Qwen3-30B-A3B, unified KV pooling reduces TTFT in long-contexts ~4.1x over state-of-the-art techniques, all making under 10 s. It also reduces blocked I/O time by up to 23.2x by eliminating filesystem overhead.

cs.AR↗

arXiv2026

Training Time Prediction for Mixed Precision-based Distributed Training

Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precision setting is a key determinant of training time, leading to training time variations of ~2.4x over its minimum. However, existing studies on distributed training time prediction rely on static model computation graphs that do not capture precision variations, including mixed precision. According to our experiments, training time prediction without considering precision results in significant prediction errors - reaching up to 147.85% in mean absolute percentage error (MAPE). To address this issue, we propose a precision-aware distributed training time predictor that achieves robust accuracy across diverse precision settings, including mixed precision, with 9.8% MAPE.

cs.LG↗

arXiv2025

GPU Memory Prediction for Multimodal Model Training

As deep learning models in agentic AI systems grow in scale and complexity, GPU memory requirements increase and often exceed the available GPU memory capacity, so that out-of-memory (OoM) errors occur. It is well known that OoM interrupts the whole training itself and wastes substantial computational resources. Therefore, to prevent OoM, accurate prediction of GPU memory usage is essential. However, previous studies focus only on unimodal architectures and fail to generalize to multimodal models, even though the multimodal models are a common choice in agentic AI systems. To address this limitation, we propose a framework that predicts the peak GPU memory usage by analyzing the model architecture and training behavior of multimodal models. Specifically, the framework decomposes the multimodal model into its constituent layers and applies factorization to estimate the memory usage of each layer. Our evaluation shows that our framework achieves high prediction accuracy of ~8.7% average MAPE.

cs.LG↗