panvram: GPU-resident random access to a human pangenome cohort

Yakiv Shavidze

Home/Archive/arXiv

arXiv · 2610.25149

panvram: GPU-resident random access to a human pangenome cohort

Yakiv Shavidze

View original source ↗Read source document ↗Export metadata ↓

Abstract

Repeatedly sampling short sequence windows from a human pangenome requires an access representation as well as compression. panvram stores each assembly as a reference-relative block archive and pools these archives on a GPU. For 558 haplotype assemblies, the default q4k representation occupies 16.642 GB of persistent pools including the decoded reference; a packed-reference, compact-directory representation occupies 11.634 GB. These quantities are not peak application memory. In the C1 experiment, default and fully compact forms delivered 743,271 and 736,762 windows/s; a separate public Colab run delivered 741,931 and 680,760 windows/s. All are batch API measurements and must not be pooled as repeated trials of an identical environment. Full-decoding checks compared reconstructed FASTA checksums with the manifest. On the CPU, the M6 shuffled-window workload at the same cohort cardinality reported 26 requests/s for AGC and 145,222 requests/s for refrel3 q4k. This comparison concerns one-thread requests of 4 KiB on the recorded warm-cache workload, not all genome-access patterns. AGC provides the smaller incremental archive: 5.40 MB/assembly versus 13.04 MB/assembly for refrel3 q16k, with different exclusions; the size pair is not the same refrel3 configuration as the q4k timing pair. We derive an expected-work expression for fixed independent granules and distinguish it from an empirical median-latency predictor. Cohort-scale counter and DRAM measurements motivate a locality hypothesis, but do not establish a causal per-miss latency or independently reproduce a model-failure verdict. The window-cost verdict at N=558 is published with its harness and raw timings.

Explore related subjects

cs.DC cs.PF q-bio.GN

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Yakiv Shavidze. 2026-10-18. panvram: GPU-resident random access to a human pangenome cohort. https://arxiv.org/abs/2610.25149

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Bitcoin-IPC: Scaling Bitcoin with a Network of Proof-of-Stake Subnets

Despite holding over $1.7T in value, Bitcoin scales poorly: Layer-1 (L1) processes only a few transactions per second, and Layer-2 (L2) solutions suffer from operational and liquidity fragmentation that requires external bridges for cross-chain connectivity. We present Bitcoin-IPC, a protocol that horizontally scales Bitcoin via permissionless, interconnected, programmable Proof-of-Stake L2 subnets staked in L1 BTC. Subnets leverage Bitcoin L1 for interconnectivity, settlement, and security, including Sybil protection, equivocating validator slashing, unilateral exit for honest stake, long-range attack protection, and compromised subnet firewalling. Bitcoin-IPC cross-subnet transfers involve only source/destination subnet validators and L1, with no external bridges or per-path liquidity lock-up. Embedded and batched in Bitcoin's witness mechanism, cross-subnet settlements can approach 6 vB per transfer, 23x less than a native L1 payment. Filling Bitcoin blocks with such batches corresponds to L1 monetary throughput of about 273 transfers per second, without modifying Bitcoin.

cs.DC↗

arXiv2026

Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods

Modern distributed optimization methods mostly rely on traditional synchronous approaches, despite substantial recent progress in asynchronous optimization. We revisit Synchronous SGD and its robust variant, called $m$-Synchronous SGD, which can be interpreted as a method with a backup-worker strategy, and theoretically show that they are nearly optimal in many heterogeneous computation and communication scenarios, which is somewhat unexpected. We analyze the synchronous methods under random times and adversarial partial participation of workers, and prove that their time complexities are optimal in many practical regimes, up to logarithmic factors. While synchronous methods are not universal solutions and there exist tasks where asynchronous methods may be necessary, we show theoretically that they are sufficient for many modern heterogeneous computation and communication scenarios.

cs.DC↗

arXiv2026

SIGMA: A Versatile Streaming Graph Partitioner for Vertex- and Edge-Balanced Distributed GNN Training

Distributed Graph Neural Network (GNN) training depends critically on how the underlying graph is partitioned across compute resources. Existing graph partitioners focus either on vertex partitioning or edge partitioning and typically optimize only a single communication objective (edge cut vertex cut) under a single balance constraint (vertex balance edge balance). We present SIGMA (Streaming Integrated Graph Partitioning with Multi-objective Awareness), a versatile streaming graph partitioner that supports both vertex and edge partitioning within a unified multi-objective, multi-constraint framework. Depending on the target distributed GNN system, SIGMA can be configured for edge-cut-oriented vertex partitioning or vertex-cut-oriented edge partitioning while simultaneously accounting for vertex and edge balancing. A clustering-based preprocessing stage incorporates global graph structure to improve partition quality while preserving the efficiency and scalability advantages of streaming partitioning. We evaluate SIGMA on six benchmark graphs spanning diverse domains and scales using two distributed GNN training systems: DistGNN (edge-partitioned) and DistDGL (vertex-partitioned). Across both settings, SIGMA consistently achieves strong performance, showing its ability to navigate complex trade-offs between partition quality, training efficiency, and memory consumption, frequently outperforming established streaming approaches and delivering competitive results against high-quality in-memory partitioners, with particularly strong results in edge-partitioned GNN training. These results demonstrate that a unified streaming partitioner can effectively address the communication, compute, and memory challenges of distributed GNN training across fundamentally different system architectures.

cs.DC↗