arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2609.31204

FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps

Mo Wang·Wenhao Ye·Zihan Ning·Jiayu Zuo·Junfeng Xia·Hongkai Wen·Quanying Liu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Can the spatial organization of cortical activity help transfer frozen image representations to fMRI prediction? We introduce FlatClip, which renders cortical activity as anatomy-linked flatmaps and extracts features with a frozen SigLIP2 encoder. Only a lightweight downstream probe is trained, without fMRI-specific encoder pretraining or fine-tuning. Across four resting-state benchmarks, FlatClip outperforms the evaluated ROI-level baselines on HCP and ADNI, but is weaker on PPMI and below the strongest voxel-level models overall. On NSD visual-fMRI decoding, restricting inputs to visual or NSD-provided task-active cortex improves recognition over whole-cortex inputs. Spatial perturbations reduce performance with both retrained and fixed readouts, and anatomy-linked arrangements outperform cortical-pixel permutations across three colormaps. Together, these results demonstrate a practical route for reusing image-pretrained features in fMRI and highlight cortical arrangement and input-region selection as important design choices. Code is available at https://github.com/OneMore1/FlatClip.

Explore related subjects

cs.CEcs.CVq-bio.NC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Mo Wang, Wenhao Ye, Zihan Ning, Jiayu Zuo, Junfeng Xia, Hongkai Wen, Quanying Liu. 2026-10-15. FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps. https://arxiv.org/abs/2609.31204

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

BioXArena: Benchmarking LLM Agents on Multi-Modal Biomedical Machine Learning Tasks

Large language model (LLM) agents increasingly automate machine-learning model building, yet biomedical benchmarks emphasize question answering, reasoning, and tool use or cover narrow slices of biomedical ML coding. We introduce BioXArena, a biomedical machine learning (BioML) coding benchmark for task-specific model building on heterogeneous, often multi-modal datasets. It contains 76 end-to-end tasks across 9 domains: sequence, single-cell, structure, network biology, chemical biology, perturbation dynamics, phenotype--disease, imaging, and text-integrated tasks. Each task packages primary-source data with hidden labels, held-out graders, and biology-aware metrics on a unified 0-to-100 scale; agents write runnable code, train models, and predict private test samples. BioXArena emphasizes realistic data interfaces: most tasks require agents to integrate multiple input sources, including 46 tasks that combine distinct modality families. We compare 18 harness--backbone configurations: general-purpose coding, biomedical, and ML agents plus our Minimal runner, all under the same controlled resource configuration. Codex with GPT-6 Astra achieves the highest task-level average score of 73.91 among 18 configurations, closely followed by Claude Code with Opus 5.5 (73.79). Matched-backbone comparisons show higher observed scores for mature coding systems than for Minimal; seven Minimal backends characterize model differences under a fixed runner. Beyond the main leaderboard, ablations, robustness checks, and scaling, runtime, and failure-mode analyses characterize how backbones, scaffolds, budgets, and domains affect BioML coding performance, exposing modeling and execution gaps across agent and LLM backbone families.

cs.CE↗
arXiv2026

CompFlowLab: A Python code to develop and prototype new data-driven models for challenging compressible flow problems with shocks and chemical reactions

CompFlowLab is an open-source Python environment that is capable of modeling different classes of compressible flow problems (including shocks, flames, and detonation waves) using a one-dimensional compressible Navier-Stokes solver with multi-species transport and chemical-reaction models. It is designed specifically for the data-driven modeling community as a lightweight, accessible prototyping platform to test, develop, and evaluate new modeling methods on numerically and physically challenging compressible flow problems, especially those featuring shocks and chemical reactions. Specifically, CompFlowLab aims at (1) providing computationally efficient calculations on advection-dominanted problems that are well-recognized to be difficult for conventional data-driven modeling techniques, such as shocks, flames, and detonation waves, and more importantly (2) enabling rapid testing and prototyping of new data-driven models. The code serves three primary purposes: (1) generating high-fidelity full order model data for training of data-driven modeling, (2) providing a modular platform for implementing and testing novel data-driven algorithms (e.g., machine learning methods and reduced-order modeling techniques) on challenging physics, and (3) offering a standalone Computational Fluid Dynamics (CFD) solver with validated test cases that can also support numerical method development in the broader CFD community. By unifying these capabilities in a clean, extensible Python codebase, CompFlowLab lowers barriers to innovation at the intersection of model reduction and complex fluid dynamics.

cs.CE↗
arXiv2026

Stochastic Latent Twins: Learning Stochastic Solution Operators

Scientific prediction under uncertainty requires more than a single future state: it requires the distribution of possible outcomes. We show that this distribution-valued evolution can be learned directly as a stochastic solution operator between arbitrary source and target times. We introduce Stochastic Latent Twins, which represent the physical state through a deterministic encoder-decoder while placing stochasticity entirely in a probability-valued latent evolution map. This separation makes the stochastic transition law independent of the state representation and enables direct probabilistic prediction without recursive time stepping. We establish an end-to-end Wasserstein approximation theory that separates representation and latent-transition errors, derive stability guarantees, and develop a distributional theory of physics conformity with latent-to-physical transfer bounds and conformity by construction. The resulting framework can be learned directly from observed transitions and supports forward prediction, backward conditional evolution, one-shot generation, and physics-conforming stochastic dynamics within the same formulation. Numerical experiments ranging from linear and non-Gaussian SDEs to stochastic Lorenz dynamics, generative modeling, a stochastic Hamiltonian system, and a two-dimensional stochastic Burgers equation verify the theory, expose the distinct roles of state representation and transition-family expressivity, compare favorably with recent stochastic and one-step generative methods, and demonstrate exact physical conservation by construction. More broadly, Stochastic Latent Twins provide a step toward scientific surrogates that represent not only the evolution of a system, but also its uncertainty and physical structure, opening new opportunities across uncertainty quantification, data assimilation, inverse problems, control, and digital twins.

cs.CE↗