When Is RTL Better? Auditing Evaluation Stability with Tool-Grounded Requirement Profiles

Bodong Hou; Zhixin Pan

Home/Archive/arXiv

arXiv · 2610.17445

When Is RTL Better? Auditing Evaluation Stability with Tool-Grounded Requirement Profiles

Bodong Hou·Zhixin Pan

View original source ↗Read source document ↗Export metadata ↓

Abstract

RTL-generation studies increasingly report functional correctness and post-synthesis quality, yet the evaluator itself is rarely treated as an experimental variable. A candidate can appear better when a security rule, protocol obligation, target context, feasibility gate, uncertainty policy, or provenance constraint is omitted. We introduce tool-grounded requirement profiles: machine-readable contracts binding each requirement's meaning to its executable evidence and comparison rule, together with cross-requirement gates, uncertainty handling, and provenance. We audit evaluator stability by freezing candidate pairs and perturbing only their evaluation profile. In a preregistered controlled pilot with six designs, twelve pairs, three requirement families, six evaluator conditions, and 72 decisions, a generic cell-count evaluator reports unsupported improvement for 6/12 pairs (50%; Wilson 95% interval 25.4 to 74.6%; design-cluster bootstrap interval 25 to 75%). The matched evaluator detects all twelve injected regressions, whereas syntax, generic synthesis, context mismatch, and gate ablation each lose the required distinction for 12/12 pairs. In a separately frozen external validation of all 290 public RTLLM v1.1 text-only candidates, 81/215 evaluable comparisons (37.7%; task-cluster 95% interval 24.8 to 51.4%) yield unsupported scalar improvements, including 72 functional-gate reversals. Separately, 26 OpenROAD-flow-scripts runs show why timing context and activity qualification must be part of the contract. The study does not rank generators; it establishes evaluator configuration as a measurable, auditable source of conclusion instability.

Explore related subjects

cs.AR cs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Bodong Hou, Zhixin Pan. 2026-10-12. When Is RTL Better? Auditing Evaluation Stability with Tool-Grounded Requirement Profiles. https://arxiv.org/abs/2610.17445

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Design

Large Language Models and vision-language models have shown remarkable success in the front-end design of Very Large-Scale Integrated Circuits, yet their capabilities for VLSI physical design remain significantly underexplored. The primary cause is the lack of standardized benchmarks for evaluating agentic physical design workflows that require high-dimensional, multi-stage optimization under strict design constraints, coordinated interaction with diverse Electronic Design Automation tools, and iterative refinement. This work introduces PDAGENT-BENCH, a comprehensive and multi-dimensional benchmark for evaluating LLM/VLM-based agents across the physical design stack. PDAGENT-BENCH integrates both task-level assessment and workflow-level execution. The benchmark suite contains 353 curated problems that combine conceptual questions with real-world industrial artifacts, with expert-validated references and executable solutions. In addition, the benchmark provides a unified, human-aligned agentic physical design workflow framework that enables closed-loop evaluation of holistic physical design in realistic EDA environments. Experiments on 11 state-of-the-art models reveal that while modern LLMs/VLMs perform competitively on conceptual tasks, they remain substantially limited in tool-centric execution (e.g., 42.2% on Innovus script generation) and long-horizon, multi-stage reasoning. Our studies further show that human-skill-enhanced agentic workflows significantly improve end-to-end physical design performance. PDAGENT-BENCH establishes a standardized, reproducible, and realistic evaluation framework for advancing LLM/VLM-driven holistic physical design automation. To ensure full reproducibility, we release \textsc{PDAgent-Bench} at https://github.com/fenggev56/PDAGENT-BENCH.

cs.AR↗

arXiv2026

FPGA Based Design And Implementation of Neural Network Compensators for Nanocrystalline MEMS Pressure Sensor

In this paper the effects of nonlinearity and temperature of nanocrystalline MEMS pressure sensor have been compensated using neural network algorithm which has been implemented on a FPGA chip. The neural network employs back propagation algorithm to train the multilayer perceptron feed forward network that causes an error reduction due to temperature upto the level of 98%. It is realized on FPGA with pipelined architecture for parallel and fast sensing operation. The pressure sensor along with the FPGA based compensation results in an intelligent sensor system capable of high speed and accurate pressure measurement which has been tested for a typical biomedical application.

cs.AR↗

arXiv2026

LizTail: Unifying Shape Adaptation and Spatial Parallelism with Segment-Reconfigurable Systolic Arrays

Systolic arrays are widely used to accelerate neural-network inference, yet fixed two-dimensional arrays struggle with the substantial diversity in both matrix geometry and available parallelism across CNNs and Transformer-based large language models. This mismatch is particularly severe for skinny matrices, small-batch autoregressive decoding, and operators containing multiple independent partitions, leading to low processing-element (PE) utilization and repeated spatial folding. Existing reconfigurable arrays typically improve either shape adaptation or distributed execution, whereas supporting both through fine-grained routing incurs considerable interconnect, buffering, and control overheads. We present LizTail (named after lizard tail autotomy, where predefined fracture planes allow segments to detach and remain independently active, analogous to LizTail's predefined reconfiguration boundaries), a multi-mode systolic array that enables constrained segment-level reconfiguration by partitioning a $128\times128$ array into 32 regular $128\times4$ segments and adding connectivity only at predefined endpoints. The resulting substrate supports trimming, long-path extension through $180^\circ$ turn-back connections, and distributed execution in which each subarray can itself be trimmed or extended. Differentiated PE datapaths and segment-aware CEB/SIB buffers provide the required compute and data-delivery flexibility without unrestricted PE-level routing. A hardware-constrained mapper selects the execution organization, dataflow, task partition, and tiling. Across representative workloads, LizTail achieves geometric-mean speedups of $5.02\times$ on CNN inference and $25.67\times$ on LLM decoding over a TPU-like array, while reducing EDP by up to $38.23\times$. LizTail is open-sourced at https://github.com/kmjbwxa/LizTail.

cs.AR↗