arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.14140

Frozen Site-Probe Accuracy Is Not a Standalone Harmonization Score: An Audit of Evaluation Practice for Multi-Site MRI

Muhammad Muqsit Islam·Gloria García Cuenco·José M. Martínez·Pierrick Coupé
View original source ↗Read source document ↗Export metadata ↓

Abstract

Image-level MRI harmonization is often evaluated with a frozen site probe, a site classifier trained on raw scans and applied unchanged to harmonized ones: a drop in its accuracy is read as removal of site effects. We audit this practice on ABIDE I (1,112 T1-weighted scans, 17 sites) with ten output sets from seven harmonization methods and controls for chance, head shape, intensity, preprocessing and target alignment. We find that this verdict depends on how the probe is trained, even among probes that classify raw scans well: across random seeds, its balanced accuracy on the same CycleGAN outputs ranges from 0.13 to 0.59 (chance: 0.06). Low frozen-probe accuracy can also arise without harmonization: a Gaussian blur, which increases the intensity distance to the target site, lowers this accuracy to chance. Moreover, where the frozen probe fails, probes trained on the harmonized scans still recover the site, even from intensity histograms alone: on HACA3 outputs, the frozen probe scores 0.25, a probe trained on those outputs 0.75, and one using only their intensity histograms 0.71. Low frozen-probe accuracy therefore does not show that site information has been removed. We recommend reporting several probes trained to convergence, probes trained on harmonized scans, and target alignment.

Explore related subjects

eess.IVcs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Muhammad Muqsit Islam, Gloria García Cuenco, José M. Martínez, Pierrick Coupé. 2026-10-09. Frozen Site-Probe Accuracy Is Not a Standalone Harmonization Score: An Audit of Evaluation Practice for Multi-Site MRI. https://arxiv.org/abs/2610.14140

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

ForcingDAS: Unified and Robust Data Assimilation via Diffusion Forcing

Data assimilation (DA) estimates the state of an evolving dynamical system from noisy, partial observations, and is widely used in scientific simulation as well as weather and climate science. In practice, filtering methods rely on frame-to-frame transition models. However, these models are fragile when observations are non-Markovian (when they form only a partial slice of a higher-dimensional latent state as in real-world weather data): they tend to accumulate errors over long horizons. At the same time, learned DA methods typically commit to a single regime, either filtering (nowcasting, real-time forecasting) or smoothing (retrospective reanalysis), which splits what should be a shared prior across application-specific pipelines. To address both issues, we introduce ForcingDAS, a unified and robust DA framework. Built on Diffusion Forcing with an independent noise level assigned to each frame, ForcingDAS learns a joint-trajectory prior instead of frame-to-frame transitions. This allows it to capture long-horizon temporal dependencies and reduce error accumulation. In addition, the same trained model spans the full filtering to smoothing spectrum at inference time. Specifically, nowcasting, fixed-lag smoothing, and batch reanalysis are selected through the inference schedule alone, without retraining. We evaluate ForcingDAS on 2D Navier-Stokes vorticity, precipitation nowcasting, and global atmospheric state estimation. Across all settings, a single model is competitive with or outperforms both learned and classical baselines that are specialized for individual regimes, with the largest gains observed on real-world weather benchmarks.

eess.IV↗
arXiv2026

Uncertainty-Gated SHAP-Dirichlet Refinement for MRI-Based Depression Classification

Structural MRI-based classification of major depressive disorder (MDD) remains challenging due to subtle, heterogeneous anatomical changes. We propose uncertainty-gated SHAP-Dirichlet refinement, a post-hoc reliability layer that improves predictions without retraining the classifier. The method retrieves similar labeled SHAP explanations, constructs Dirichlet class evidence, and selectively refines uncertain predictions through probabilistic fusion, while allowing unreliable cases to be deferred for expert review. We evaluate five classifiers on 504 SRPBS subjects (252 controls and 252 MDD patients) using 159 anatomical features and repeated nested cross-validation. Refinement improved accuracy across all 8 repetitions for TabM and ResNet, yielding mean gains of 1.04 and 0.45 percentage points (sign test p=0.008), while modifying at most 8.3% of predictions. Gains were concentrated among low-confidence cases, with significant improvements in calibration. The method also benefited an RBF support vector machine but not logistic regression or random forests. Ablations confirmed the importance of uncertainty gating and soft fusion. Dirichlet evidence identified baseline errors (AUROC 0.62), while deferring 50% of cases increased retained-case accuracy to approximately 75%. These findings support selective explanation-guided refinement as a practical approach to improving prediction reliability and facilitating expert triage in MDD classification.

eess.IV↗
arXiv2026

Image CL-SEC: Cross-Layer Semantic Error Correction for Image Communication

Semantic error correction (SEC) uses the semantic knowledge of pretrained models to complement conventional channel decoding. By combining physical-layer decoding information with application-layer semantic predictions, cross-layer SEC (CL-SEC) has demonstrated substantial performance gains in text transmission; however, its applicability to image transmission remains largely unexplored. This paper proposes Image CL-SEC, which, to the best of our knowledge, is the first extension of the CL-SEC framework to image transmission. Image CL-SEC uses channel-decoding information together with contextual predictions from a pretrained visual model to resolve residual uncertainty after forward error correction (FEC) decoding. In our implementation, images are represented as discrete visual tokens. For each token position selected for re-evaluation, the probability distribution produced by the channel decoder is combined with the distribution predicted by the visual model to identify the most likely transmitted token. The resulting decision therefore reflects both channel evidence and visual context. Experiments across multiple channel conditions show that, at low and moderate signal-to-noise ratios (SNRs), Image CL-SEC improves pixel-level fidelity, perceptual similarity, and token-recovery accuracy relative to either channel decoding or visual-model inference alone. Supplementary experiments using different pretrained visual models further demonstrate that these gains are not specific to a single model.

eess.IV↗