arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.13027

Dataset-Specific Encoding of Image Content Underlies Cross-Dataset Failure in Brain MRI Foundation Models

Minh Sao Khue Luu·Bair N. Tuchinov
View original source ↗Read source document ↗Export metadata ↓

Abstract

Frozen features of brain MRI foundation models lose accuracy on new datasets, a failure usually blamed on the features encoding where each scan came from. We show that the failure lies elsewhere. Across 17 public datasets and eight frozen representations, every representation recognizes the MRI sequence within each dataset, yet most fail on an unseen one. The cause is neither dataset encoding nor a shift of each dataset: the content itself is encoded along different directions in each dataset. This explains which datasets fail, for the sequence and for sex, and why removing the dataset signal with INLP, LEACE, CORAL or adversarial training never improves sequence transfer. Only the broadly pretrained 3DINO and SAM-Med3D encode the sequence consistently enough to transfer; the brain-specific BrainIAC and BrainFM do no better than an untrained network. Code is available at https://github.com/BrainFM/brainfm-profiler/tree/main/representation_utility.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Minh Sao Khue Luu, Bair N. Tuchinov. 2026-10-09. Dataset-Specific Encoding of Image Content Underlies Cross-Dataset Failure in Brain MRI Foundation Models. https://arxiv.org/abs/2610.13027

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence

In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features carrying semantically meaningful information. Pioneering works in that direction have shown that modelling 3D objects by ellipsoids and 2D detections by ellipses offers a convenient manner to link 2D and 3D data. However, the mathematical formalism most often used in the related litterature does not enable to easily distinguish ellipsoids and ellipses from other quadrics and conics, leading to a loss of specificity potentially detrimental in some developments. Moreover, the linearization process of the projection equation creates an over-representation of the camera parameters, also possibly causing an efficiency loss. In this paper, we therefore introduce an ellipsoid-specific theoretical framework and demonstrate its beneficial properties in the context of pose estimation. More precisely, we first show that the proposed formalism enables to reduce the pose estimation problem to a position or orientation-only estimation problem in which the remaining unknowns can be derived in closed-form. Then, we demonstrate that it can be further reduced to a 1 Degree-of-Freedom (1DoF) problem and provide the analytical derivations of the pose as a function of that unique scalar unknown. We illustrate our theoretical considerations by visual examples and include a discussion on the practical aspects. Finally, we release this paper along with the corresponding source code in order to contribute towards more efficient resolutions of ellipsoid-related pose estimation problems.

cs.CV↗
arXiv2026

Optimising for the Unknown: Domain Alignment for Cephalometric Landmark Detection

Cephalometric Landmark Detection is the process of identifying key areas for cephalometry. Each landmark is a single GT point labelled by a clinician. A machine learning model predicts the probability locus of a landmark represented by a heatmap. This work, for the 2024 CL-Detection MICCAI Challenge, proposes a domain alignment strategy with a regional facial extraction module and an X-ray artefact augmentation procedure. The challenge ranks our method's results as the best in MRE of 1.186mm and third in the 2mm SDR of 82.04% on the online validation leaderboard. The code is available at https://github.com/Julian-Wyatt/OptimisingfortheUnknown.

cs.CV↗
arXiv2026

Understanding Transformer-based Vision Models through Inversion

Understanding the internal mechanisms of deep neural networks remains a central challenge in machine learning. In computer vision, one promising yet only preliminarily explored approach is feature inversion via inverse networks, which reconstructs images from intermediate representations using trained inverse networks. In this study, we revisit feature inversion via inverse networks, introducing a novel, modular variant that enables a computationally more efficient application of the technique while, in some cases, producing semantically more coherent image reconstructions. We apply our method to large-scale transformer-based vision models, specifically Detection Transformer, Vision Transformer, Swin Transformer, and Data-Efficient Image Transformer, analyzing the resulting reconstructions across network depth. Our main analysis compares Detection Transformer and Vision Transformer, which exhibit the most informative differences among the evaluated architectures. At the same time, results for Swin Transformer and Data-Efficient Image Transformer support the broader applicability of our framework. Our findings reveal gradual representational changes across transformer layers as a shared characteristic of Detection Transformer and Vision Transformer, as well as systematic differences in how they preserve contextual shape and fine-grained image details, and in their robustness to color perturbations. These findings deepen understanding of transformer-based vision models and demonstrate the utility of modular feature inversion as an interpretability tool.

cs.CV↗