arXiv Science⌕ Search

arXiv subjects

Weiling Yang

Publications and source records attributed to Weiling Yang.

10 recordsLinked to original sources

PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE Inference

Local deployment of MoE-based LLMs on CPU-GPU nodes is attractive for cost and privacy reasons, yet limited GPU memory forces most experts into CPU memory. Prefill is then compute-bound and benefits from aggressive GPU loading, while decode is I/O-bound and requires conservative PCIe transfers. These demands cannot be amortized under low batching---typical of local assistants and document tools---forcing phase-agnostic schedulers into a seesaw: favoring prefill's aggressive loading injects unmaskable I/O that stalls decode, while favoring decode's conservative transfers starves prefill's GPU of work. We observe that expert routing is predictable in phase-specific ways, allowing us to decouple the two phases and optimize prediction, scheduling, and execution independently for each phase. PDScope predicts expert activations to guide placement and prefetching, adapts these policies to each phase's bottleneck via cross-layer cost modeling, and compiles decisions into overhead-free pipelines using a static computation graph, cache prefetching, and zero-copy asynchronous I/O. On mainstream MoE models, PDScope achieves 2.5-10.3x higher prefill throughput and 2.1-6.1x higher decode throughput than existing systems, showing that phase-aware co-design unlocks efficient MoE inference under memory constraints and extends naturally to multi-batch settings.

cs.DC↗

HiNa-MoE: High-Performance, Non-Intrusive MoE Inference on CPUs with Matrix Engines

Mixture-of-Experts (MoE) inference is increasingly deployed in local and on-premise environments, where expert parameters often exceed GPU memory capacity. In latency-sensitive, low-concurrency settings, repeatedly staging routed-expert weights from CPU memory to the GPU can be prohibitive, leaving routed-expert feed-forward networks (FFNs) on the critical path of multi-socket CPUs. Existing CPU accelerations often rely on intrusive, hardware- or topology-specific requirements, such as AMX-specific weight layouts or manual NUMA-aware placement. These requirements reduce portability and complicate integration with standard CPU-GPU offloading pipelines. We present HiNa-MoE, a high-performance, non-intrusive operator library for MoE inference on CPUs with Intel AMX. HiNa-MoE (1) exploits AMX with an optimized micro-kernel that keeps expert weights in standard layouts and instead fuses lightweight layout transforms into token gathering and stores; (2) applies NUMA-aware task partitioning under a simple page-interleaved policy without modifying the framework allocator; and (3) converts decode-phase memory matrix-vector operations into small matrix-matrix execution to utilize AMX. Across multiple MoE models, HiNa-MoE achieves up to 3.37x speedup for FFN kernels and up to 2.09x end-to-end inference speedup over state-of-the-art baselines, while remaining plug-and-play with existing frameworks and deployment workflows.

cs.DC↗

LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers

Mixture-of-Experts (MoE) models face memory and PCIe latency bottlenecks when deployed on commodity hardware. Offloading expert weights to CPU memory results in PCIe transfer latency that exceeds GPU computation by several folds. We present PreScope, a prediction-driven expert scheduling system that addresses three key challenges: inaccurate activation prediction, PCIe bandwidth competition, and cross-device scheduling complexity. Our solution includes: 1) Learnable Layer-Aware Predictor (LLaPor) that captures layer-specific expert activation patterns; 2) Prefetch-Aware Cross-Layer Scheduling (PreSched) that generates globally optimal plans balancing prefetching costs and loading overhead; 3) Asynchronous I/O Optimizer (AsyncIO) that decouples I/O from computation, eliminating waiting bubbles. PreScope achieves 141% higher throughput and 74.6% lower latency than state-of-the-art solutions.

cs.LG↗

Demystifying ARM SME to Optimize General Matrix Multiplications

General Matrix Multiplication (GEMM) is a critical kernel in high-performance computing and deep learning. While modern architectures like ARM's Scalable Matrix Extension (SME) introduce dedicated hardware for matrix operations, existing linear algebra libraries fail to fully exploit its potential, particularly for large matrices. This paper presents MpGEMM, an open-source library that leverages key architectural features of SME to optimize GEMM across multiple precisions. Through a systematic characterization of SME, we derive optimization guidelines that inform our design. MpGEMM employs cache-aware partitioning, efficient data packing with on-the-fly transposition, and specialized micro-kernels that utilize multi-vector loads and all available tile registers. Evaluated on an Apple M4 Pro with real-world workloads from DeepSeek and LLaMA, MpGEMM achieves an average speedup of 1.23x over the vendor-optimized Apple Accelerate library and significantly outperforms other open-source alternatives.

cs.DC↗

A recursive definition for the polymatroid Tutte polynomial

The Tutte polynomial is a significant invariant of graphs and matroids. It is well-known that it has three equivalent definitions: bases expansion, rank generating function, and deletion-contraction formula. The polymatroid Tutte polynomial $\mathscr{T}_{P}$ generalizes the Tutte polynomial from matroids to polymatroids $P$. In \emph{[Adv. Math. 402 (2022) 108355.]} and \emph{[J. Combin. Theory Ser. A 188 (2022) 105584]}, the authors provided bases expansion and rank generating function constructions for $\mathscr{T}_{P}$, respectively. In \emph{[Int. Math. Res. Not. 19 (2025) rnaf302]}, a recursive formula for $\mathscr{T}_{P}$ was obtained. In this paper, we show that the recursive formula itself can be used to define the polymatroid Tutte polynomial independently.

math.CO↗

A sharp upper bound for the number of connected sets in any grid graph

A connected set in a graph is a subset of vertices whose induced subgraph is connected. Although counting the number of connected sets in a graph is generally a \#P-complete problem, it remains an active area of research. In 2020, Vince posed the problem of finding a formula for the number of connected sets in the $(n\times n)$-grid graph. In this paper, we establish a sharp upper bound for the number of connected sets in any grid graph by using multistep recurrence formulas, which further derives enumeration formulas for the numbers of connected sets in $(3\times n)$- and $(4\times n)$-grid graphs, thus solving a special case of the general problem posed by Vince. In the process, we also determine the number of connected sets of $K_{m}\times P_{n}$ by employing the transfer matrix method, where $K_{m}\times P_{n}$ is the Cartesian product of the complete graph of order $m$ and the path of order $n$.

math.CO↗

Isolation partitions in graphs

Let $G$ be a graph and $k \geq 3$ an integer. A subset $D \subseteq V(G)$ is a $k$-clique (resp., cycle) isolating set of $G$ if $G-N[D]$ contains no $k$-clique (resp., cycle). In this paper, we prove that every connected graph with maximum degree at most $k$, except $k$-clique, can be partitioned into $k+1$ disjoint $k$-clique isolating sets, and that every connected claw-free subcubic graph, except 3-cycle, can be partitioned into four disjoint cycle isolating sets. As a consequence of the first result, every $k$-regular graph can be partitioned into $k+1$ disjoint $k$-clique isolating sets.

math.CO↗

Extreme coefficients of multiplicity Tutte polynomials

The multiplicity Tutte polynomial, which includes the arithmetic Tutte polynomial, is a generalization of the classical Tutte polynomial of matroids. In this paper, we obtain an expression of the general coefficient and the expressions of six extreme coefficients of multiplicity Tutte polynomials. In particular, an expression of the general coefficient and the expressions of corresponding extreme coefficients of classical Tutte polynomial of matroids are deduced.

math.CO↗

Convolution formulas for multivariate arithmetic Tutte polynomials

The multivariate arithmetic Tutte polynomial of arithmetic matroids is a generalization of the multivariate Tutte polynomial of matroids. In this note, we give the convolution formulas for the multivariate arithmetic Tutte polynomial of the product of two arithmetic matroids. In particular, the convolution formulas for the multivariate arithmetic Tutte polynomial of an arithmetic matroid are obtained. Applying our results, several known convolution formulas including [5, Theorem 10.9 and Corollary 10.10] and [1, Theorems 1 and 4] are proved by a purely combinatorial proof. The proofs presented here are significantly shorter than the previous ones. In addition, we obtain a convolution formula for the characteristic polynomial of an arithmetic matroid.

math.CO↗

On the polymatroid Tutte polynomial

The Tutte polynomial is a well-studied invariant of matroids. The polymatroid Tutte polynomial $\mathcal{J}_{P}(x,y)$, introduced by Bernardi et al., is an extension of the classical Tutte polynomial from matroids to polymatroids $P$. In this paper, we first prove that $\mathcal{J}_{P}(x,t)$ and $\mathcal{J}_{P}(t,y)$ are interpolating for any fixed real number $t\geq 1$, and then we study the coefficients of high-order terms in $\mathcal{J}_{P}(x,1)$ and $\mathcal{J}_{P}(1,y)$. These results generalize results on interior and exterior polynomials of hypergraphs.

math.CO↗