arXiv ScienceSearch

arXiv subjects

Sa Wang

Publications and source records attributed to Sa Wang.

At least 19 recordsLinked to original sources

Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing energy consumption. Existing energy-management approaches adapt GPU frequencies only at the request or inference-phase level, overlooking operator-level differences in frequency sensitivity between Attention and feed-forward networks (FFNs). We find that the energy-optimal frequencies of Attention and FFN (A/F) differ and vary with the inference phase, workload, and system configurations. However, runtime variability and independent A/F frequency control create a large search space and high communication overhead. To address these challenges, we present AFlex, a framework that jointly optimizes resource provisioning and GPU frequency scaling for disaggregated A/F serving. AFlex introduces a global scheduler and a local operator-level dynamic voltage and frequency scaling (DVFS) controller to determine A/F resource allocations and frequencies. It further introduces an interleaved A/F pipeline with dynamic microbatch depth and adaptive request batching to reduce pipeline bubbles. We implement AFlex in SGLang and evaluate it on NVIDIA A800 GPUs using Qwen3-32B and Mixtral-8$\times$7B under production Conversation and Coding traces. \AFlex reduces energy per token by up to 49\% over state-of-the-art disaggregated serving and 48\% over frequency-scaling systems while satisfying TTFT and TPOT SLOs.

cs.DC

SwiftCache: Efficient LLM Serving for Multi-turn Conversations with Heterogeneous KV Cache Sharing

Multi-turn conversation is a fundamental scenario in LLM applications, widely used in chatbots and AI agents. As the conversation evolves, historical tokens accumulate continuously. Existing systems cache their key-value (KV) pairs to avoid redundant computation. However, limited GPU memory (HBM) capacity often forces these KV caches to be offloaded to CPU memory or SSD, making KV cache reloads increasingly costly in terms of latency as the context grows. Meanwhile, the constrained HBM capacity also limits the maximum inference length, thereby restricting the number of turns that can be supported in a conversation. To address these two challenges, we propose SwiftCache, a collaborative inference system that enables heterogeneous models to share underutilized GPU memory and NVLink bandwidth within a server. Specifically, models with low KV cache demand donate idle GPU memory to store the prefix cache of high-demand models, allowing cross-model KV cache sharing over NVLink and avoiding slow PCIe transfers. SwiftCache further reduces memory pressure by keeping only the KV cache of the currently active layer in local GPU memory, thereby enabling longer-context inference. Our experiments on real-world workloads show that SwiftCache reduces P99 time-to-first-token (TTFT) by up to 69% and extends maximum context length by up to 3.98x compared to vLLM and SGLang, with minimal interference to co-located models.

cs.DC

Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda

The rapid rise of Large Language Models (LLMs) has revolutionized various artificial intelligence (AI) applications, from natural language processing to code generation. However, the computational demands of these models, particularly in training and inference, present significant challenges. Traditional systems are often unable to meet these requirements, necessitating the integration of cloud-native and distributed architectures. This paper explores the role of cloud platforms and distributed systems in supporting the scalability, efficiency, and optimization of LLMs. We discuss the complexities of LLM deployment, including data management, resource optimization, and the need for microservices, autoscaling, and hybrid cloud-edge solutions. Additionally, we examine emerging research trends, such as serverless inference, quantum computing, and federated learning, and their potential to drive the next phase of LLM innovation. The paper concludes with a roadmap for future developments, emphasizing the need for continued research, standardization, and cross-sector collaboration to sustain the growth of LLMs in both research and enterprise applications.

cs.DC

Heavy-quark transport across the QCD crossover driven by a lattice-constrained in-medium potential

We present a self-consistent framework for heavy-quark transport in the quark-gluon plasma across the QCD crossover region. By synthesizing perturbative and nonperturbative interactions into a unified interaction kernel, we circumvent the traditional reliance on arbitrary soft-hard momentum separation scales. The interaction is governed by an in-medium effective potential, incorporating short-range Yukawa screening and long-range confining string contributions, both rigorously constrained by the latest lattice QCD data. Our results reveal that the nonperturbative string tension is indispensable for capturing the extreme opacity of the medium near the critical temperature $T_c$. Specifically, our model predicts a spatial diffusion coefficient of $2\pi T D_s \approx 0.5 \sim 1.7$, demonstrating a striking quantitative agreement with the recent lattice QCD extractions. Ultimately, our results provide a robust dynamical interpretation of the strong heavy-quark coupling near the QCD crossover and offer a unified framework for describing heavy-flavor transport in hot and dense QCD matter.

hep-ph

UCAgent: An End-to-End Agent for Block-Level Functional Verification

Functional verification remains a critical bottleneck in modern IC development cycles, accounting for approximately 70% of total development time in many projects. However, traditional methods, including constrained-random and formal verification, struggle to keep pace with the growing complexity of modern semiconductor designs. While recent advances in Large Language Models (LLMs) have shown promise in code generation and task automation, significant challenges hinder the realization of end-to-end functional verification automation. These challenges include (i) limited accuracy in generating Verilog/SystemVerilog verification code, (ii) the fragility of LLMs when executing complex, multi-step verification workflows, and (iii) the difficulty of maintaining verification consistency across specifications, coverage models, and test cases throughout the workflow. To address these challenges, we propose UCAgent, an end-to-end agent that automates hardware block-level functional verification based on three core mechanisms. First, we establish a pure Python verification environment using Picker and Toffee to avoid relying on LLM-generated SystemVerilog verification code. Second, we introduce a configurable 31-stage fine-grained verification workflow to guide the LLM, where each stage is verified by an automated checker. Furthermore, we propose a Verification Consistency Labeling Mechanism (VCLM) that assigns hierarchical labels to LLM-generated artifacts, improving the reliability and traceability of verification. Experimental results show that UCAgent can complete end-to-end automated verification on multiple modules, including the UART, FPU, and integer divider modules, achieving up to 98.5% code coverage and up to 100% functional coverage. UCAgent also discovers previously unidentified design defects in realistic designs, demonstrating its practical potential.

cs.SE

Lyra: A Hardware-Accelerated RISC-V Verification Framework with Generative Model-Based Processor Fuzzing

As processor designs grow more complex, verification remains bottlenecked by slow software simulation and low-quality random test stimuli. Recent research has applied software fuzzers to hardware verification, but these rely on semantically blind random mutations that may generate shallow, low-quality stimuli unable to explore complex behaviors. These limitations result in slow coverage convergence and prohibitively high verification costs. In this paper, we present Lyra, a heterogeneous RISC-V verification framework that addresses both challenges by pairing hardware-accelerated verification with an ISA-aware generative model. Lyra executes the DUT and reference model concurrently on an FPGA SoC, enabling high-throughput differential checking and hardware-level coverage collection. Instead of creating verification stimuli randomly or through simple mutations, we train a domain-specialized generative model, LyraGen, with inherent semantic awareness to generate high-quality, semantically rich instruction sequences. Empirical results show Lyra achieves up to $1.27\times$ higher coverage and accelerates end-to-end verification by up to $107\times$ to $3343\times$ compared to state-of-the-art software fuzzers, while consistently demonstrating lower convergence difficulty.

cs.AR

RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference

The quadratic complexity of attention mechanisms poses a critical bottleneck for large language models processing long contexts. While dynamic sparse attention methods offer input-adaptive efficiency, they face fundamental trade-offs: requiring preprocessing, lacking global evaluation, violating query independence, or incurring high computational overhead. We present RRAttention, a novel dynamic sparse attention method that simultaneously achieves all desirable properties through a head \underline{r}ound-\underline{r}obin (RR) sampling strategy. By rotating query sampling positions across attention heads within each stride, RRAttention maintains query independence while enabling efficient global pattern discovery with stride-level aggregation. Our method reduces complexity from $O(L^2)$ to $O(L^2/S^2)$ and employs adaptive Top-$τ$ selection for optimal sparsity. Extensive experiments on natural language understanding (HELMET) and multimodal video comprehension (Video-MME) demonstrate that RRAttention recovers over 99\% of full attention performance while computing only half of the attention blocks, achieving 2.4$\times$ speedup at 128K context length and outperforming existing dynamic sparse attention methods.

cs.CL

TurboFuzz: FPGA Accelerated Hardware Fuzzing for Processor Agile Verification

Verification is a critical process for ensuring the correctness of modern processors. The increasing complexity of processor designs and the emergence of new instruction set architectures (ISAs) like RISC-V have created demands for more agile and efficient verification methodologies, particularly regarding verification efficiency and faster coverage convergence. While simulation-based approaches now attempt to incorporate advanced software testing techniques such as fuzzing to improve coverage, they face significant limitations when applied to processor verification, notably poor performance and inadequate test case quality. Hardware-accelerated solutions using FPGA or ASIC platforms have tried to address these issues, yet they struggle with challenges including host-FPGA communication overhead, inefficient test pattern generation, and suboptimal implementation of the entire multi-step verification process. In this paper, we present TurboFuzz, an end-to-end hardware-accelerated verification framework that implements the entire Test Generation-Simulation-Coverage Feedback loop on a single FPGA for modern processor verification. TurboFuzz enhances test quality through optimized test case (seed) control flow, efficient inter-seed scheduling, and hybrid fuzzer integration, thereby improving coverage and execution efficiency. Additionally, it employs a feedback-driven generation mechanism to accelerate coverage convergence. Experimental results show that TurboFuzz achieves up to 2.23x more coverage collection than software-based fuzzers within the same time budget, and up to 571x performance speedup when detecting real-world issues, while maintaining full visibility and debugging capabilities with moderate area overhead.

cs.AR

Unveiling the jet angular broadening with photon-tagged jets in high-energy nuclear collisions

The medium modification of jet substructure in hot and dense nuclear matter has garnered significant interest from the heavy-ion physics community in recent years. Measurements of inclusive jets show an angular narrowing in nucleus-nucleus collisions, while recent CMS results for photon-tagged jets ($γ$+jets) suggest evidence of broadening. In this study, we conduct a theoretical analysis of the angular structure of inclusive jets and $γ$+jets using a transport approach that accounts for jet energy loss and the medium response in the quark-gluon plasma. We examine the girth modification of $γ$+jets in $0-30\%$ PbPb collisions at $\sqrt{s_{NN}} = 5.02$ TeV, achieving satisfactory agreement with recent CMS measurements. We explore the relationship between selection bias and jet kinematics by varying the threshold for $x_{jγ} = p_T^{\rm jet}/p_T^γ$. Notably, we quantitatively demonstrate that $γ$+jets significantly reduce selection bias and can effectively select jets that have been sufficiently quenched in PbPb collisions, which is crucial for capture the jet angular broadening. Additionally, we estimate the contributions of medium-induced gluon radiation and the medium response to the broadening of the jet angular substructure. Lastly, we analyze the modification patterns of jet $R_g$ and $ΔR_{\rm axis}$ in PbPb collisions, which indicate slight broadening for $γ$+jets and noticeable narrowing for inclusive jets compared to pp collisions.

hep-ph

Substructure grooming of inclusive and photon-tagged jets in heavy-ion collisions

Jet substructure provides a powerful probe of partonic interactions within the quark-gluon plasma (QGP) in heavy-ion collisions. In this paper, we present a systematic theoretical study of the groomed substructures for both inclusive jets and photon-tagged jets ($γ+$jets) utilizing the Dynamical and Soft-Drop Grooming algorithms in PbPb collisions by employing the SHELL transport model. Our theoretical calculations exhibit a suppression at high $k_{\rm T,g}$, the relative transverse momentum between the two subjets in the groomed substructure, consistent with the recent ALICE measurements. We show that the suppression of high $k_{\rm T,g}$ arises from the combined effects of the reduction of the subleading subjet transverse momentum due to partonic energy loss and the narrowing of the groomed jet radius $R_g$ induced by selection bias. Our findings demonstrate that no enhancement is observed at high $k_{\rm T,g}$, even in the complete absence of selection bias. Furthermore, we propose that the broadening of $R_g$ in photon-tagged jets, which are less susceptible to selection bias compared to inclusive jets, provides relatively direct evidence of the jet substructure broadening. Our analysis reveals that the $R_g$ broadening becomes more pronounced as the jet radius increases, where the medium-induced gluon radiation plays a dominant role in driving such broadening. In particular, we find that as the jet radius increases, the Soft Drop grooming algorithm exhibits a better resolving power for the contribution of the medium response to the jet substructure broadening.

hep-ph

Soft-hard factorization of heavy-quark transport in QCD matter at finite chemical potential

We calculate the collisional energy loss and momentum diffusion coefficients of heavy quarks traversing a hot and dense QCD medium at finite quark chemical potential, $μ\neq0$. The analysis is performed within an extended soft-hard factorization model (SHFM) that consistently incorporates the $μ$-dependence of the Debye screening mass $M_D(μ)$ and of the fermionic thermal distribution functions. Both the energy loss and the diffusion coefficients are found to increase with $μ$, with the enhancement being most pronounced at low temperatures where the chemical potential effects dominate the medium response. To elucidate the origin of this dependence, we derive analytic high-energy approximations in which the leading $μ$-corrections appear as logarithmic terms: a soft logarithm $\simμ^{2}\ln(|t^{*}|/M_{D}^{2})$ from $t$-channel scattering off thermal gluonic excitations, and a hard logarithm $\simμ^{2}\ln(E_{1}T/|t^{*}|)$ from scattering off thermal quarks. In the complete result the dependence on the intermediate separation scale $t^{\ast}$ cancels, as required. We also confirm the expected mass hierarchy $-dE/dz(charm)<-dE/dz(bottom)$ at fixed velocity. Our findings demonstrate that finite chemical potential plays a significant role in heavy-quark transport and must be included in theoretical descriptions of heavy-flavor dynamics in baryon-rich environments, such as those probed in the RHIC Beam Energy Scan, and at FAIR and NICA.

hep-ph

Perturbative and nonperturbative properties of heavy quark transport in a thermal SU(3) gluon plasma

We investigate the perturbative and nonperturbative aspects of heavy quark transport in a thermal SU(3) gluon plasma. Based on the soft-hard factorized model, we extend the original perturbative framework to the near-critical temperature region, where nonperturbative effects become significant. The transition behavior of the semi-quark-gluon-plasma (semi-QGP) is described via a temperature-dependent background field incorporated in the background field effective theory. By implementing this approach, we quantitatively evaluate the collisional energy loss and momentum diffusion coefficients of charm and bottom quarks as functions of the incoming energy and medium temperature. Our results show a distinct suppression of both the energy loss and the diffusion coefficients relative to conventional perturbative estimates, especially near the critical temperature. This suppression originates from the emergence of a temperature-dependent color background field, which effectively reduces the color charge screening of the medium. These findings provide important theoretical insight into the phenomenology of heavy-flavor probes, offering a unified theoretical framework applicable across both high- and low-momentum regimes.

hep-ph

Multijet topology in high-energy nuclear collisions: jet broadening

This work presents the first theoretical investigation of the medium modification of jet broadening as an event-shape observable in multijet final states due to jet quenching in high-energy nuclear collisions. The partonic spectrum of $pp$ collisions with next-to-leading order (NLO) accuracy at $\sqrt{s_{\mathrm{NN}}} = 5.02$ TeV is provided by the POWHEG$+$PYTHIA8 event generator, while the linear Boltzmann transport (LBT) model is utilized to investigate the energy loss of fast partons as they traverse through the hot and dense QCD medium. We present the jet broadening distributions in multijet final states for both $pp$ and PbPb collisions at $\sqrt{s_{\mathrm{NN}}} = 5.02$ TeV, then observe an enhancement at the small jet broadening region and suppression at the large jet broadening region in PbPb collisions relative to that in $pp$. This suggests that medium modification with parton energy loss in the QGP leads to a more concentrated energy flow in all observed multijet events in PbPb reactions. We also demonstrate that the intertwining of two effects, the jet number reduction and the restructured contribution, results in the novel behavior of nuclear modification of the jet broadening observable in PbPb collisions.

nucl-th

DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving

The Text-to-Video (T2V) model aims to generate dynamic and expressive videos from textual prompts. The generation pipeline typically involves multiple modules, such as language encoder, Diffusion Transformer (DiT), and Variational Autoencoders (VAE). Existing serving systems often rely on monolithic model deployment, while overlooking the distinct characteristics of each module, leading to inefficient GPU utilization. In addition, DiT exhibits varying performance gains across different resolutions and degrees of parallelism, and significant optimization potential remains unexplored. To address these problems, we present DDiT, a flexible system that integrates both inter-phase and intra-phase optimizations. DDiT focuses on two key metrics: optimal degree of parallelism, which prevents excessive parallelism for specific resolutions, and starvation time, which quantifies the sacrifice of each request. To this end, DDiT introduces a decoupled control mechanism to minimize the computational inefficiency caused by imbalances in the degree of parallelism between the DiT and VAE phases. It also designs a greedy resource allocation algorithm with a novel scheduling mechanism that operates at the single-step granularity, enabling dynamic and timely resource scaling. Our evaluation on the T5 encoder, OpenSora SDDiT, and OpenSora VAE models across diverse datasets reveals that DDiT significantly outperforms state-of-the-art baselines by up to 1.44x in p99 latency and 1.43x in average latency.

cs.DC

Phenomenological study of the angle between jet axes in heavy-ion collisions

This paper presents a phenomenological study on the angle between the Standard and the Winner-Take-All (WTA) jet axes ($ΔR_{\rm axis}^{\rm WTA-Std}$) in high-energy nuclear collisions. The $p$+$p$ baseline is provided by the Pythia8 event generator. The in-medium jet propagation is simulated by the linear Boltzmann transport (LBT) model, which considers both the elastic and inelastic jet-medium interactions. Our theoretical results calculated by the LBT model show that the $ΔR_{\rm axis}^{\rm WTA-Std}$ distribution in Pb+Pb at $\sqrt{s}=5.02$ TeV is narrower than that in $p$+$p$, which agrees well with the recent ALICE measurements. The narrowing of $ΔR_{\rm axis}^{\rm WTA-Std}$ seems to violate the $p_T$-broadening nature of the jet quenching effect, usually explained by the influence of "selection bias". However, the physical details still need to be fully understood. Utilizing a matching-jet method to track the jet evolution in the QGP to remove the selection bias in the Monte Carlo simulations, we observe that the $ΔR_{\rm axis}^{\rm WTA-Std}$ distribution becomes broader due to the jet-medium interactions. At the same time, by rescaling the quark/gluon-jet fractions in Pb+Pb collisions to be the same as that in $p$+$p$, we find that the fraction change may not significantly influence the modification pattern of jet $ΔR_{\rm axis}^{\rm WTA-Std}$. On the other hand, the selected jet sample in A+A collisions has a significantly narrower initial $ΔR_{\rm axis}^{\rm WTA-Std}$ distribution than the $p$+$p$ baseline, and such a biased comparison between $p$+$p$ and A+A conceals the actual jet-broadening effect in the experimental measurements. The investigations presented in this paper will deepen our understanding of the relationship between the actual intra-jet modifications in the QGP and the experimental observations.

hep-ph

Production of leptons from decay of heavy-flavor hadrons in high-energy nuclear collisions

This paper presents a theoretical study on the production of the heavy-flavour decay lepton (HFL) in high-energy nuclear collisions at the LHC. The pp-baseline is calculated by the FONLL program, which matches the next-to-leading order pQCD calculation with the next-to-leading-log large-$p_T$ resummation. The in-medium propagation of heavy quarks is driven by the modified Langevin equations, which consider both the elastic and inelastic partonic interactions. We propose a method to separate the respective influence of the six factors, such as pp-spectra, the cold nuclear matter (CNM) effects, in-medium energy loss (E-loss), fragmentation functions (FFs), coalescence (Coal), and decay channels, which may contribute to the larger $R_{AA}$ of HFL $\leftarrow b$ compared to that of HFL $\leftarrow c$ in nucleus-nucleus collisions. Based on quantitative analysis, we demonstrate that both coalescence hadronization, decay channels and the mass-dependent E-loss play an essential role at $p_T<5$ GeV, while the latter dominates the higher $p_T$ region. It is also found that the influences of the CNM effects and FFs are insignificant. At the same time, different initial pp-spectra of charm and bottom quarks have a considerable impact at $p_T>5$ GeV. Furthermore, we explore the path-length dependence of jet quenching by comparing the HFL $R_{AA}$ in two different collision systems. Our investigations show smaller HFL $R_{AA}$ in Pb+Pb than in Xe+Xe within the same centrality bin, consistent with the ALICE data. The longer propagation time and more effective energy loss of heavy quarks in Pb+Pb collisions play critical roles in the stronger yield suppression of the HFL compared to that in Xe+Xe. In addition, we observe a scaling behavior of the HFL $R_{AA}$ in Xe+Xe and Pb+Pb collisions.

hep-ph

Probing the mass effect of heavy quark jets in high-energy nuclear collisions

The production of heavy quark (HQ) jets provides a new arena to address the mass effect of jet quenching in heavy-ion physics. This paper presents a theoretical study of HQ jet yield suppression in Pb+Pb collisions at the LHC and focuses on the energy loss of HQ jets produced by different mechanisms. The p+p baseline is carried out by the SHERPA generator, and the jet-medium interactions are described by the SHELL transport model, which considers the elastic and inelastic partonic energy loss in the quark-gluon plasma (QGP). In p+p collisions, our numerical results indicate that the HQ jets from gluon splitting ($g \rightarrow Q$-jet) give the dominant contribution at high $p_T$, and it shows more dispersive structures than the HQ-initiated one ($Q \rightarrow Q$-jet). In nucleus-nucleus collisions, our calculations are consistent with the inclusive and b-jet $R_{AA}$ recently measured by the ATLAS collaboration, which suggests a remarkable manifestation of the mass effect of jet energy loss. As a result of the dispersive substructure, the $g \rightarrow Q$-jet will lose more energy than the $Q \rightarrow Q$-jet in the QGP. Due to the significant contribution of $g \rightarrow c$-jet, the $R_{AA}$ of c-jet will be comparable or even smaller than that of inclusive jet. To experimentally distinguish the $g \rightarrow Q$-jet and $Q \rightarrow Q$-jet, we propose the event selection strategies based on their topological features and test the performances. By isolating the $c \rightarrow c$-jet and $b \rightarrow b$-jet, the jets initiated by heavy quarks, we predict that the order of their $R_{AA}$ are in line with the mass hierarchy of energy loss. Future measurements on the $R_{AA}$ of $Q \rightarrow Q$-jet and $g \rightarrow Q$-jet will provide a unique chance to test the flavor/mass dependence of energy loss at the jet level.

hep-ph

Medium modifications of heavy-flavor jet angularities in high-energy nuclear collisions

We present the first theoretical study of heavy-flavor jet angularities ($λ^κ_α$) in Pb+Pb collisions at $\sqrt{s_{\rm NN}}=$ 5.02 TeV. The initial production of heavy-flavor jets is carried out using the POWHEG+PYTHIA8 prescription, while the jet evolution in the quark-gluon plasma (QGP) is described by the SHELL transport model. In p+p collisions, we observe narrower angularity distributions for D$^0$-tagged jets compared to inclusive jets, consistent with the ALICE preliminary results. We then demonstrate that jet quenching in the QGP slightly widens the angularity distribution of D$^0$-tagged jets in Pb+Pb collisions relative to that in p+p collisions for jet transverse momentum of $10 < p_{\rm T,jet} < 20$ GeV/c, while the angularity distributions of inclusive and D$^0$-tagged jets become narrower in Pb+Pb collisions relative to p+p at $p_{\rm T,jet} > 20$ GeV/c due to the strong influence of the selection bias. Additionally, by comparing the average angularities $\langle λ^κ_α \rangle$ of inclusive, D$^0$-tagged and B$^0$-tagged jets with varying $α$ and $κ$, we show that the larger the quark mass is, the lower the jet's $\langle λ^κ_α \rangle$ values are. As a result of the slenderer initial distribution, we predict that as compared to inclusive jets, the heavy-flavor jets, especially the B$^0$-tagged ones, will suffer stronger modifications of $\langle λ^κ_α \rangle$ in Pb+Pb relative to p+p at $10 < p_{\rm T,jet} < 20$ GeV/c. For a larger jet radius, a more significant broadening of jet angularities is predicted because of the enhanced contributions of the wide-angle particles.

hep-ph