William Oliveira | arXiv Science

arXiv2026

Throughput per Megabyte: A Pilot Benchmark of Language-Stack Efficiency for Self-Hosted HTTP Services on a Raspberry Pi 5

Cloud-centric web benchmarks miss constraints that matter for self-hosted services on ARM64 single-board computers, especially idle RAM footprint and energy per request. We ran a pilot benchmark on one Raspberry Pi 5, measuring equivalent SQLite-backed CRUD APIs implemented in Go 1.26/net/http, Rust 1.95/Axum, Python 3.13/FastAPI+Granian, Node.js 24/Fastify, and .NET 10 Native AOT across N=50 randomized runs per stack and endpoint, with a separate concurrency sweep and memory time-series collection. Rust achieved the highest weighted throughput-to-RAM ratio (310.25 req/s/MB, 2.8x Go) because its weighted peak RSS was 7.36 MB, while .NET, Node.js, Go, and Rust formed a raw-throughput cluster with overlapping bootstrap CIs (2,125--2,461 req/s) and Python remained below that cluster at 969 req/s. The concurrency sweep on GET /items/:id showed divergent scaling: .NET reached 26,209 req/s at c=240, Go and Rust plateaued near c=120, Node.js saturated near 9,700 req/s, and Python stayed below 2,300 req/s; memory snapshots showed Rust holding a flat 7.8 MB RSS while Node.js grew by 117 MB without visible GC drops. For RAM-constrained self-hosted CRUD-over-SQLite deployments on this Pi 5 unit, the data support Rust as the most resource-efficient measured stack, .NET as the strongest high-concurrency stack, and Go as a practical option where ecosystem breadth matters; replication is needed before extending these findings to other devices or storage backends.

cs.PF↗

arXiv2026

Less Is More: Engineering Challenges of On-Device Small Language Model Integration in a Mobile Application

On-device Small Language Models (SLMs) promise fully offline, private AI experiences for mobile users (no cloud dependency, no data leaving the device). But is this promise achievable in practice? This paper presents a longitudinal practitioner case study documenting the engineering challenges of integrating SLMs (Gemma 4 E2B, 2.6B parameters; Qwen3 0.6B, 600M parameters) into Palabrita, a production Android word-guessing game. Over a 5-day development sprint comprising 204 commits (~90 directly AI-related), the system underwent a radical transformation: from an ambitious design where the LLM generated complete structured puzzles (word, category, difficulty, and five hints as JSON) to a pragmatic architecture where curated word lists provide the words and the LLM generates only three short hints, with a deterministic fallback if it fails. We identify five categories of failures specific to on-device SLM integration: output format violations, constraint violations, context quality degradation, latency incompatibility, and model selection instability. For each failure category, we document the observed symptoms, root causes, and the prompt engineering and architectural strategies that effectively mitigated them, including multi-layer defensive parsing, contextual retry with failure feedback, session rotation, progressive prompt hardening, and systematic responsibility reduction. Our findings demonstrate that on-device SLMs are viable for production mobile applications, but only when the developer accepts a fundamental constraint: the most reliable on-device LLM feature is one where the LLM does the least. We distill our experience into eight actionable design heuristics for practitioners integrating SLMs into mobile apps.

cs.SE↗