arXiv Science⌕ Search

arXiv subjects

Aditya Chatterjee

Publications and source records attributed to Aditya Chatterjee.

5 recordsLinked to original sources

High-Throughput Serving of Nemotron 3.5 Transcribe with Regime-Aware Megakernels and Kernel Fusion

Nemotron 3.5 Transcribe ranks first among open-weight models on the Hugging Face Open ASR leaderboards. Serving it efficiently takes more than fast operators: continuous batching changes the amount and mix of work at every iteration, and the best fusion granularity changes with it. We present a serving engine that, for every pass, chooses between persistent megakernels and chains of fused kernels at crossovers measured offline. On an H100 SXM GPU, throughput of our accuracy-preserving serving peaks at 4,345x real time, 1.39x the peak of tuned vLLM, and is up to 4.39x faster at low concurrency; on B200 it peaks at 6,367x, 1.64x vLLM at the highest concurrency we measured. Word error rate (WER) is 1.91 to 1.93% against 1.88 to 1.93% for vLLM. Within a 400 ms budget for p95 end-to-end completion time, it serves 8x as many concurrent requests. Choosing per pass beats either level alone: 23% higher throughput than always using megakernels at 128 concurrent requests, and 9% higher than always using fused kernels at 8. Opt-in switches, including greedy MoE layer skipping, add 13 to 16% throughput on B200 for offline jobs that can accept higher WER.

cs.PF↗

Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies

Vision-Language-Action (VLA) policies translate language and visual inputs into robot actions, where their hidden representations directly shape closed-loop behavior. However, mechanistic interpretability tools from language and vision-language models do not transfer cleanly to VLAs: outputs are robot actions rather than human-readable tokens, and validating internal changes requires expensive closed-loop rollouts. We propose an event-grounded interpretability pipeline that anchors SAE feature analysis to behavioral events rather than text contexts. We extract keyframes from end-effector trajectories and cluster them within each task using visual, robot-state, and temporal cues. The resulting events guide SAE feature ranking, with optional VLM annotations providing descriptions of the events. To our knowledge, our pipeline is among the first to ground SAE-based VLA analysis in closed-loop behavioral events. Across two simulation architectures and a real-robot study, event-grounded ranking yields the strongest causal effects on OpenVLA and transfers to the continuous action chunks of $π_{0.5}$. SAE is a sparse but imperfect intervention basis: usability varies with architecture and intervention site, and aggressive intervention reveals safety and interpretability limits. Overall, event-grounded SAE analysis emerges as a practical starting point for behavior-anchored VLA interpretability, motivating future work on SAE analysis beyond end-effector motion, finer-grained closed-loop evaluation, and safe interventions for high-stakes VLA deployments.

cs.RO↗

One Megakernel is Not Enough: Predicting Where to Cut for Streaming Inference

We address a fundamental question: How far should kernel fusion extend and which boundaries are worth preserving? Unlike the general convention of using one monolith megakernel at lower batch sizes or supporting only a selective model component as megakernel and rest of the model as conventional graph, we show that the best configuration is a composition of multiple megakernels with no overlapping conventional graph component, or a fully conventional graph. We present a framework that selects megakernel regions in a streaming inference model from conventional-graph profiles and standalone kernel benchmarks, without running any candidate design end to end. It simulates each candidate's tile and event graphs to predict time to first output and selects the measured best design in all 13 combinations of model and batch size we tested. Across NH-TTS, Qwen3-TTS and Llama-1B, the best composition ranges from two to four megakernels to fully conventional execution and none is a single megakernel. We introduce four model categories (megakernel-stable/ receding/ advancing/ averse) that capture how a model's best megakernel composition changes with batch size. Using NH-TTS model on H100 NVL GPU, we demonstrate that preserving three specialized megakernels can improve both latency and throughput despite introducing additional kernel boundaries. At batch size 32, this design delivers 21.4% lower mean batch-p95 time to first audio (TTFA) and 28% higher throughput than the evaluated monolith megakernel. Against conventional TensorRT graph execution, it reduces engine-level mean batch TTFA p95 by 4.05x and increases throughput by 19.7%.

cs.PF↗

From Roofline to Ruggedness: Decomposing and Smoothing the GEMM Performance Landscape

Adjacent GEMM problems that differ by a single 128-element step in N can show 30% different throughput. This pervasive performance ruggedness - invisible to roofline analysis and peak-FLOPs intuition, yet dominant for every non-peak workload - is the subject of this paper. We propose performance ruggedness analysis, an analytical framework complementary to roofline: rather than summarizing a GPU with a scalar bound, it treats the full multidimensional performance surface as the object of study, decomposes its texture into mechanism-attributable components, and separates software-removable from hardware-bound losses. The framing is analogous to deep-learning loss landscapes: a continuous quantity (idealized time 2MNK/peak) made rugged by discrete hardware substrates (tiles, sub-groups, cache lines, DRAM channels). We instantiate it on BF16 NN GEMM on Intel Battlemage (Arc B580, sycl-tla) via a 32,768-configuration sweep over (M,N,K) in {128,...,4096}^3. We introduce roughness, the mean absolute step-to-step throughput change, which starts at 16.8 TFLOPs/128-step against an ideal of 2.0. A two-stage stack - best-of-six dynamic tile selection and a novel dynamic-programming padding-and-splitting optimizer (precomputed once, O(1) at runtime) - cuts roughness by 70% and raises mean throughput by 30%. Cross-tile experiments show the residual sawtooth period scales exactly with the tile size, ruling out cache conflicts and attributing the rest to four hardware-bound sources. Finally, we derive the optimal achievable landscape from first principles - datasheet integers alone, no kernel run or simulator - and turn it into an optimality scale (Kernel Optimality Levels) grading any kernel by how much of that landscape it attains and how close its roughness lies to the hardware floor; the production kernel and our optimized stack rate L0 and L2 despite both reporting ~95% of peak.

cs.PF↗

Bias corrected estimators for proportion of true null hypotheses under exponential model: Application of adaptive FDR-controlling in segmented failure data

Two recently introduced model based bias corrected estimators for proportion of true null hypotheses ($π_0$) under multiple hypotheses testing scenario have been restructured for exponentially distributed random observations available for each of the common hypotheses. Based on stochastic ordering, a new motivation behind formulation of some related estimators for $π_0$ is given. The reduction of bias for the model based estimators are theoretically justified and algorithms for computing the estimators are also presented. The estimators are also used to formulate a popular adaptive multiple testing procedure. Extensive numerical study supports superiority of the bias corrected estimators. We also point out the adverse effect of using the model based bias correction method without proper assessment of the underlying distribution. A case-study is done with a synthetic dataset in connection with reliability and warranty studies to demonstrate the applicability of the procedure, under a non-Gaussian set up. The results obtained are in line with the intuition and experience of the subject expert. An intriguing discussion has been attempted to conclude the article that also indicates the future scope of study.

math.ST↗