arXiv Science⌕ Search

arXiv subjects

Jacob M. Ryan

Publications and source records attributed to Jacob M. Ryan.

2 recordsLinked to original sources

BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing

Looped architectures scale quality while keeping parameter count fixed by repeatedly passing activations through the same block of layers. During inference, each recurrence generates independent KV-caches, causing memory to grow linearly. To address this challenge, prior work has proposed KV-sharing (i.e., sharing KV-caches across depth), finding that, in some cases, the KV-cache of the final recurrence can be shared with the earlier recurrences without quality loss, a property we denote as KV-shareability. However, the mechanisms underlying KV-shareability remain poorly understood, with even identically trained looped architectures not consistently exhibiting it. In this paper, we study how KV-shareability can be enabled in looped models. We propose a two-stage method: in the first stage, we use stochastic depth during pretraining, which induces contractive residual dynamics that drive representations toward a common limit across recurrences. In the second stage, we propose BeNice!, a post-training method that broadcasts the final recurrence's KV-cache into the first recurrence, training it to produce compatible residuals while preserving the original model's quality. As a result, BeNice! enables KV-shareability while matching standard finetuning's unshared validation perplexity. Using BeNice!, we show that KV-sharing is complementary to alternative KV-compression methods, composing with memory-efficient sequence mixers (e.g., multi-latent attention, Kimi delta attention) and low-precision KV-cache quantization (e.g., fp8, mxfp8, nvfp4). Finally, in simulated low-memory, long-context inference, KV-sharing improves cache hit rates by up to 31.4%, increases throughput by up to 23.6%, and reduces p90 time-to-first-token by up to 83.6%.

cs.LG↗

Diffusion in heterogeneous discs and spheres: new closed-form expressions for exit times and homogenization formulae

Mathematical models of diffusive transport underpin our understanding of chemical, biochemical and biological transport phenomena. Analysis of such models often focusses on relatively simple geometries and deals with diffusion through highly idealised homogeneous media. In contrast, practical applications of diffusive transport theory inevitably involve dealing with more complicated geometries as well as dealing with heterogeneous media. One of the most fundamental properties of diffusive transport is the concept of mean particle lifetime or mean exit time, which are particular applications of the concept of first passage time, and provide the mean time required for a diffusing particle to reach an absorbing boundary. Most formal analysis of mean particle lifetime applies to relatively simple geometries, often with homogeneous (spatially-invariant) material properties. In this work, we present a general framework that provides exact mathematical insight into the mean particle lifetime, and higher moments of particle lifetime, for point particles diffusing in heterogeneous discs and spheres with radial symmetry. Our analysis applies to geometries with an arbitrary number and arrangement of distinct layers, where transport in each layer is characterised by a distinct diffusivity. We obtain exact closed-form expressions for the mean particle lifetime for a diffusing particle released at an arbitrary location and we generalise these results to give exact, closed-form expressions for any higher-order moment of particle lifetime for a range of different boundary conditions. Finally, using these results we construct new homogenization formulae that provide an accurate simplified description of diffusion through heterogeneous discs and spheres.

physics.bio-ph↗