arXiv ScienceSearch

arXiv subjects

Mike Helwig

Publications and source records attributed to Mike Helwig.

2 recordsLinked to original sources

Memory as Infrastructure: Reliability Engineering for Persistent Agent Memory in Months-Long LLM-Assisted Development

LLM coding agents are crossing from task-scale to project-scale: single assisted efforts that run for months, across repeated context compactions, on codebases far larger than any context window. We report operational experience from one such effort (a research project under a single continuous Claude Code session line since January 2026, driving a 633,000-line codebase, with its memory subsystem continuously instrumented since July 2026) and describe SIx Harness, the open-source memory and continuity infrastructure that emerged from keeping it coherent. The harness combines per-project long-term memory (hybrid lexical-vector retrieval over SQLite, fully local), precision-gated context injection, anti-recurrence stores for decisions and dead-ends, conventions engineered to survive compaction, and, the part we argue is missing from operational practice, reliability engineering for the memory subsystem itself: a session-start health gate with discriminated failure modes, heartbeat telemetry designed so that no enumerated failure mode can pass unrecorded, and alert-fatigue budgeting borrowed from SRE practice. We present what we believe is the first months-scale instrumented operational record of a persistent agent-memory subsystem in production development use: 78,933 hook invocations; 85 recorded failures, none silent: 84 in the subsystem's first three weeks, one since, none in the final 20 days; an injection layer whose ten-day precision instrument shows zero false fires against an intact denominator; and three production incidents traced from instrument reading to structural fix. From the record we distill seven design principles, state our limitations plainly (N=1, no control arm, self-reported), and publish a tagged pre-registered ablation protocol that any team can run with the released MIT-licensed kit.

cs.SE

Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses

Text-to-SQL benchmarks ship schemas whose column names already say what the columns mean. Production warehouses are the inverse: cryptic identifiers, partial or absent documentation. We address the problem they pose first: recovering what columns and values mean from the data itself. Rosetta places a language model inside a verification harness: a deterministic profiler extracts structural evidence (value fingerprints, a 26-pattern library, checksum verdicts), the model proposes semantics conditioned on that evidence, and every fact carries provenance and a confidence bounded by its evidence class. Against human documentation on 680 paired columns across eleven BIRD databases, identifiers destroyed, the harness delivers metadata that is 0.475 accurate on the 42% of columns it commits to, against 0.223 on 94% for the same model used directly. Restricted to the 283 columns where both arms speak, the harness writes no better prose than the model alone; the gain is selection: deterministic evidence governs whether the system speaks (coverage +0.257 [0.128, 0.378]), not how well. The deterministic layer is a competence detector, not a competence amplifier. A backbone swap bounds the claim: the prose finding reproduces, but prompt-requested abstention does not transfer; a code-enforced commit gate (predictions registered first; measured on a third backbone and held-out databases) makes no-evidence coverage 0.000 on every backbone. On a blind i2b2 clinical warehouse Rosetta decodes 95.5% of 134 real ICD-9 codes from values alone and abstains on all 44 NDC drug codes. The catalog supports calibrated abstention at query time: under full schema opacity a naive translator falls from 0.92 to 0.42 execution accuracy while our gate answers at 86% accuracy over 59% coverage. Negative results are reported plainly, including that our own authority ladder is not the mechanism behind the headline.

cs.DB