arXiv Science⌕ Search

arXiv · cs/9903014

Perpetual Adaptation of Software to Hardware: An Extensible Architecture for Providing Code Optimization as a Central System Service

Abstract

We present an open architecture for just-in-time code generation and dynamic code optimization that is flexible, customizable, and extensible. While previous research has primarily investigated functional aspects of such a system, architectural aspects have so far remained unexplored. In this paper, we argue that these properties are important to generate optimal code for a variety of hardware architectures and different processor generations within processor families. These properties are also important to make system-level code generation useful in practice.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Thomas Kistler, Michael Franz. 1999-03-22. Perpetual Adaptation of Software to Hardware: An Extensible Architecture for Providing Code Optimization as a Central System Service. https://arxiv.science/abs/cs/9903014

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

HyCoInfer: Cloud-Edge Collaborative Inference for Large Language Models on Heterogeneous GPU-FPGA Architectures

Large language model (LLM) serving relies predominantly on cloud GPUs, yet autoregressive decoding is memory-bound and often leaves their arithmetic units underutilized. Edge FPGAs offer attractive decoding efficiency, but are ill suited to sustaining a high-concurrency serving workload alone. This paper presents HyCoInfer, a heterogeneous cloud-edge serving system that assigns prefill to cloud GPUs and distributes decoding across cloud GPUs and edge FPGAs. The design addresses three barriers to practical offloading: capacity balancing across heterogeneous accelerators, the large KV cache crossing a constrained wide-area link, and the latency added by heterogeneous execution. HyCoInfer incorporates FPGA decode capacity into GPU prefill/decode role partitioning and places requests according to queue load, prompt length, and network state. It further combines layer-wise pipelined transfer, decoder-side precomputation, and bandwidth-adaptive KV cache compression based on residual vector quantization. We implement HyCoInfer using two NVIDIA V100 GPUs and an AMD Xilinx Alveo U280 and evaluate it over 100-1000 Mbps links. At 500 Mbps, HyCoInfer delivers up to 4.11 times the request goodput of cloud-only vLLM and reduces mean end-to-end latency by up to 61.1%. Across different network conditions and workloads, the system improves overall serving capacity, with the largest gains on communication-intensive tasks, and reduces per-request energy.

cs.OS↗

Bounded Local Generator Classes for Deterministic State Evolution

Persistent memory must retain history without revisiting it at every transition. Bounded Local Generator Classes (BLGC) formalize carried numerical state, selected context and ordered local updates. The $\ell^\infty$ formulation permits countably many uniformly bounded coordinates without square summability. Projected maps preserve admissibility, Lipschitz estimates control finite-horizon sensitivity, and bounded acquisition and evaluation give a cardinality-independent transition-work contract. OPAL supplies a working persistent-state engine and controlled production measurements. After six successful million-node workloads, 18 matched fresh Release processes compared original and incremental receipt indexing at 10,000, 100,000 and 1,000,000 nodes. Degree, record shape, state dimension, active-target count and event schedule were fixed. Repaired resident-update P50 remained 0.083-0.084 microseconds; subsequent receipt-event P50 ranged from 4.458 to 6.459 microseconds across all scales. The original receipt path took 177.700-182.297 milliseconds at one million nodes. Reliability was tested alongside performance. Baseline and repaired engines produced byte-identical checkpoints, including continuation after fresh-process recovery. Replay, conflict, rejection, concurrent-event and allocation-failure checks passed; all 18 matched scaling processes passed their correctness gates. The repair preserves the state law and durable receipt format. These results connect the mathematical construction to a traced production transition with verified state continuity and measured scaling. Capacity-growth costs and formal scope are reported separately.

cs.OS↗

The Compute ICE-AGE: OPAL ONE, Continuity That Persists While Reasoning Is Idle

The Compute ICE-AGE separates persistent application memory from probabilistic reasoning. We present OPAL, a CPU-resident C++17 graph engine that preserves record identities, relationships, and accepted numerical state between reasoning sessions. Applications evolve that state through defined updates, save canonical checkpoints, and recover continuity after restart. Bounded Local Generator Classes (BLGC) provide the mathematical foundation for this bounded local evolution. On Apple Silicon M2 Pro hardware, the current-engine resident-update P50 measured 0.083 microseconds at 100,000, 1,000,000, and 5,000,000 retained Atomic Memory Vertices (AMVs). A same-session 1M-5M-1M sequence reproduced that median using one frozen binary and 1,024 prepared targets, with every process passing its correctness checks. These measurements cover complete resident-update calls, including validation, indexed lookup, and numerical state mutation. Initialization and storage-growth costs are reported separately. Reliability tests verified event replay, conflict rejection, concurrent calls, allocation-failure handling, and fresh-process recovery with continued operation. Engine-version comparisons produced byte-identical checkpoints and continuation files. The installed PYAMV Python package passed 17 SDK contract checks and 20 workload checks. KINDRED's Model Context Protocol interface passed 22 distinct focused checks covering memory operations, authorization, and transport. Together, these results demonstrate persistent continuity with low recurring local-update costs through five million retained AMVs, providing working interfaces for applications and developers. The architecture keeps accepted memory available while reasoning is idle, allowing applications to invoke models only when tasks require interpretation, planning, or generation.

cs.OS↗