arXiv subjects

Jack Li

Publications and source records attributed to Jack Li.

6 recordsLinked to original sources

arXiv2026

flash-comm: Self-Synchronizing Collective Communication for LLM Decoding

We present flash-comm, a communication library that achieves efficient transport for LLM decoding by fusing readiness synchronization with data movement. flash-comm introduces self-synchronizing transfers in which each data block carries evidence of its readiness, allowing receivers to directly validate the blocks they need without separate inter-GPU barriers. Based on this principle, a cost-guided protocol selector adapts the readiness encoding to message size and the number of participating GPUs, balancing synchronization overhead against additional data-transfer cost. flash-comm further decouples transport order from reduction order to preserve result consistency across GPUs under asynchronous data arrivals. On a server with eight NVIDIA B300 GPUs, flash-comm reduces all-reduce latency by a median of 28.8% and up to 43.7% compared with custom AR v2 (the state-of-the-art all-reduce implementation). In end-to-end decoding, flash-comm reduces time per output token (TPOT) by 1.7-6.8%. Note that all-reduce accounts for 11.4% of the baseline decoding time, which bounds the latency reduction possible from optimizing this operation alone. The same mechanism also supports standalone all-gather, reducing latency by a median of 23.8% and up to 58.9% across eight B300 GPUs.

cs.DC↗

arXiv2026

UlaCache: Unified Logical Addressing for Sparse LLM Serving

Long-context inference in large language models (LLMs) imposes substantial memory requirements for the key-value (KV) cache. Token-selective sparse attention reduces attention computation and memory traffic, but KV entries must still be retained for future queries. KV storage capacity therefore remains a constraint on decoding concurrency. Combining GPU and CPU memory expands this capacity but requires efficient access across heterogeneous memory pools. We present UlaCache, a KV cache management system that provides unified logical addressing across GPU and CPU memory. Its logical-slot interface enables the inference engine to allocate, write, access, and release KV entries using stable identifiers across attention parallelism modes. UlaCache manages physical placement, residency, and transfers, updating address mappings after KV transfers. Parallel Bucketed LRU (PB-LRU) reduces the overhead of KV replacement by parallelizing recency updates and eviction-candidate selection within a GPU memory pool shared across requests. We evaluate UlaCache against state-of-the-art baselines. Under tensor-parallel attention (TP), UlaCache provides up to $11.4\times$ the usable KV cache capacity of HiSparse. For GLM-5.3 on eight H200 GPUs, UlaCache achieves peak decoding throughput of $1.10\times$ and $1.59\times$ that of HiSparse under data-parallel attention (DPA) and TP, respectively. For HY4 Preview on eight B300 GPUs, the corresponding throughput ratios are $1.27\times$ and $1.93\times$.

cs.DC↗

arXiv2026

Repair, Retry, and the Cost of Hindsight: Verifier Feedback in Formal Theorem Proving

Verifier-guided self-repair conditions a model on feedback from failed attempts before it tries again, without weight updates. Yet two findings in prior work are in tension: verifier feedback can improve repair of failures, but self-repair itself can provide little-to-no advantage over independent resampling. We reconcile these findings in Lean theorem proving through controlled one-step experiments that hold the preceding failure fixed while varying whether the failed proof is shown, whether the compiler error corresponds to that failure, and the retry instruction. We test frozen Qwen3.6-27B, a general-purpose model, and DeepSeek-Prover-V2-7B, a specialized prover, on difficult MiniF2F problems. We find two opposing effects. Showing the model its failed predecessor reduces success and sharply increases copying of it, while on Qwen adding that attempt's compiler error recovers the loss, to the level of the diagnostic alone. Replacing the predecessor with another theorem's failed proof largely removes the degradation on both models; on Qwen, replacing the diagnostic with another problem's error substantially reduces the recovery. Without the failed proof, correct diagnostics show no resolved advantage over wrong-diagnostic controls on either model. The failed-proof penalty persists with and without the retry instruction, whereas Qwen's diagnostic recovery is weaker without it. In 11-problem sequential experiments through 64 attempts, Qwen retains a correct-diagnostic benefit within failed-proof histories, while DeepSeek-Prover shows no resolved recovery. However, no tested one-step feedback condition establishes an advantage over instruction-only resampling, nor does 64-attempt sequential repair with failed proofs and correct diagnostics at equal token budgets. Verifier feedback can thus improve repair without overcoming the performance loss from conditioning on a failed attempt.

cs.AI↗

arXiv2026

FlashBoot: Sub-Second Weight Loading for Large Models at Rack Scale

Flagship Mixture-of-Experts (MoE) models are growing fast along two axes at once: total parameter count and the number of experts. In elastic deployment scenarios, many GPUs across many nodes must become serving-ready quickly, and this growth makes weight loading a noticeable part of the latency budget. Even on NVIDIA's GB300 NVL72, today's state-of-the-art loaders leave most of that bandwidth unused. The losses are structural: (C1) weight memory is fragmented into tens of thousands of per-tensor objects, so transfers run far below link bandwidth; (C2) cross-node replication is gated by NCCL communicator setup, which costs 10-110 s before a single weight byte moves; and (C3) the existing cross-node GPU->GPU clone path is serial and scales poorly to concurrent multi-node bring-up. We present FlashBoot, a hardware-friendly, framework-workflow co-designed weight-loading subsystem built on SGLang. At its core is FabricArena, a contiguous, exportable and inter-node addressable tensor memory layout. On top of it, FlashLoad loads from CPU as a single bulk, zero-copy transfer, and FlashClone replicates a resident model from a remote GPU via a remote-mapping mechanism that removes NCCL setup. In experiments on NVL72 with DeepSeek-V4-Pro and DeepSeek-V4-Flash, FlashClone maps remote weight memory in ~10 ms (versus 10-110 s for NCCL) and sustains >=700 GB/s per clone. Against the state of the art, FlashBoot accelerates single-node weight loading by up to 50x (from 20.1 s to 0.4 s) and concurrent rack-level weight loading by >270x (from 87 s to 0.32 s). Our code will be made publicly available.

cs.DC↗

arXiv2018

Characterizing the Quantum Confined Stark Effect in Semiconductor Quantum Dots and Nanorods for Single-Molecule Electrophysiology

We optimized the performance of quantum confined Stark effect QCSE based voltage nanosensors. A high throughput approach for single particle QCSE characterization was developed and utilized to screen a library of such nanosensors. Type II ZnSe CdS seeded nanorods were found to have the best performance among the different nanosensors evaluated in this work. The degree of correlation between intensity changes and spectral changes of the excitons emission under applied field was characterized. An upper limit for the temporal response of individual ZnSe CdS nanorods to voltage modulation was characterized by high throughput, high temporal resolution intensity measurements using a novel photon counting camera. The measured 3.5 us response time is limited by the voltage modulation electronics and represents about 30 times higher bandwidth than needed for recording an action potential in a neuron.

cond-mat.mes-hall↗

arXiv2018

Muon Spin Rotation Analysis of the Internal Magnetic Field of Heavy Fermion System Uranium Beryllium-13

Uranium beryllium-13 is a heavy fermion system whose anomalous behavior may be explained by its poorly understood internal magnetic structure. Here, uranium beryllium-13's magnetic distribution is probed via muon spin spectroscopy ($μ$SR)-a process where positive muons localize at magnetically unique sites in the crystal lattice and precess at characteristic Larmor frequencies, providing measurements of the internal field. Muon spin experiments using the transverse-field technique conducted at varying temperatures and external magnetic field strengths are analyzed via statistical methods on ROOT. Two precession frequencies are observed at low temperatures with an amplitude ratio in the Fourier transform of 2:1, enabling muon stopping sites to be traced at the geometric centers of the edges of the crystal lattice. Characteristic strong and weak magnetic sites are deduced, additionally verified by mathematical relationships. Results can readily be applied to other heavy fermion systems, and recent identification of quantum critical points in a host of heavy fermion compounds show a promising future for the application of these systems in quantum technology. Note that this paper is an analysis of data, and all experiments mentioned here are conducted by a third party.

cond-mat.mtrl-sci↗