An Integrated Framework for Rule-Based Data Editing

Pierpaolo Massoli

Home/Archive/arXiv

arXiv · 2610.16947

An Integrated Framework for Rule-Based Data Editing

Pierpaolo Massoli

View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper develops a mathematical framework connecting structural metadata, logical admissibility, and correction in statistical data editing. A normalized routing graph generates a canonical indexed representation and an associated Horn theory, incorporating explicit semantic edits through deterministically evaluated auxiliary variables. Under specified record semantics, the theory characterizes admissibility, distinguishes structural minimality from logical irredundancy, and permits validation without materializing the deductive closure. Canonical indices preserve the provenance and dependency scopes of violations. With complete diagnostics and strictly positive weights, a nonnegative violation Hamiltonian vanishes exactly on admissible corrections within the permitted candidate domain. Verified zero-energy outputs therefore certify feasibility and global minimization of violation energy. Minimum modification cost is formulated separately, with sufficient conditions for an equivalent penalized objective and optimality certificates obtained by matching attained costs with valid lower bounds. A household survey application verifies admissible corrections and certifies minimum cost for a subset of returned records through a retrospective audit. The framework connects structural requirements to correction certificates independently of the numerical search procedure, while statistical accuracy of replacement values requires separate assessment.

Explore related subjects

stat.ME

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Pierpaolo Massoli. 2026-10-12. An Integrated Framework for Rule-Based Data Editing. https://arxiv.org/abs/2610.16947

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Assumption-Lean Differential Variance Inference for Heterogeneous Treatment Effect Detection

The conditional average treatment effect (CATE) is frequently estimated in clinical studies to refute a homogeneous treatment effect hypothesis. Under this regime, all patients making up the population experience identical benefit from a given treatment relative to a comparator. Uncovering heterogeneous treatment effects through inference about the CATE, however, requires that covariates truly modifying the treatment effect be reliably collected at baseline. CATE-based techniques will necessarily fail to detect violations when effect modifiers are omitted from the data due to, for example, resource constraints. Severe measurement error has a similar impact. Clinical decision makers can be misled as a result. To address these limitations, we prove that a practical homogeneous treatment effect hypothesis can be gauged through inference about contrasts of the potential outcomes' variances even when effect modifiers are missing from the data. We derive causal machine learning estimators of these contrasts and study their asymptotic properties. We establish that these estimators are doubly robust and asymptotically linear under mild conditions, permitting formal hypothesis testing about treatment effect heterogeneity. Numerical experiments demonstrate that these estimators' asymptotic guarantees are approximately achieved in finite-sample randomized and observational study data alike. These inference procedures are then used to detect heterogeneous treatment effects in the re-analysis of randomized controlled trials investigating targeted temperature management in cardiac arrest patients.

stat.ME↗

arXiv2026

Estimating marginal association between paired outcomes in clustered data with an application to analyzing caries and periodontal disease data from the National Health and Nutrition Examination Survey

The within-cluster subgroup composition of units can inform on the marginal distribution of those units outcomes, denoted informative subgroup size (ISS). Moreover, within-cluster subgroup composition of paired outcomes can allow one outcome to inform upon the marginal distribution of the other. If this type of ISS ignored, it can bias marginal association estimates by allowing subgroup composition to influence how clusters contribute to the analysis. We develop two weighting approaches for estimating marginal association between paired outcomes in this setting. The weights are derived using within-cluster resampling arguments and extend conventional cluster weighting to paired response subgroups. We adapt an existing test of marginal dependence to assess ISS by examining this subgroup structure in both directions. In simulated data exhibiting this bidirectional ISS, pair-based weighting reduced bias when association was primarily carried through the paired observations. These weights also allow scientifically or clinically meaningful dichotomizations to enter directly into the weighting scheme, with the choice of subgroup definition further improving estimation in several simulated settings. Application to NHANES oral-health data showed strong evidence of ISS for filled-surface outcomes across subgroups defined by clinical attachment loss and pocket depth, with the corresponding periodontal--caries associations generally small and positive. These methods provide a practical approach for estimating marginal association when paired subgroup composition is informative.

stat.ME↗

arXiv2026

Functional Principal Component Analysis for Manifold-Indexed Data

Functional principal component analysis has traditionally focused on processes indexed by Euclidean domains, although functional observations increasingly arise on curved domains. We develop FPCA for real-valued processes observed with noise at discrete locations on a general Riemannian manifold. We estimate the mean and covariance functions using intrinsic kernel smoothing based on geodesic distance and a normal-coordinate volume correction, with a general weighting scheme accommodating unequal numbers of observations across subjects. We establish $L^2$ and uniform convergence rates for these estimators and derive corresponding error bounds for the estimated eigenvalues and eigenfunctions. These results quantify how manifold dimension, sampling frequency and weighting affect estimation accuracy and characterize the phase transition between sparse and dense sampling regimes. Simulation studies on the circle and sphere, together with real-data applications, illustrate the finite-sample performance of the proposed methodology.

stat.ME↗