arXiv subjects

Marc-Andre Schulz

Publications and source records attributed to Marc-Andre Schulz.

8 recordsLinked to original sources

arXiv2026

Prior Action Control Causally Alters Language-Model Plasticity

Two copies of one language model start from identical weights and receive the same rewards during training. For one copy the rewards depend on its own answers. The other copy is yoked: it receives the first copy's rewards no matter what it answers, a design taken from the learned-helplessness experiments of animal learning. We ask whether the copy whose rewards depended on its answers, the copy with control, later learns a new task faster, that is, whether control changes the model's plasticity. In Experiment 1, copies of Qwen3-0.6B generated their own answers in a two-turn game and afterwards learned a new task. The copies with control learned the new task faster in every pair, and the advantage held on different starting models as well as on the larger Qwen3-1.7B. Copies with no reward signal, trained only on the updates that every copy shared, learned about as fast as the yoked copies. The advantage is therefore mostly a benefit of control, not damage from lacking it. Control had also changed how the copies answered, so the twins practiced different answers. That practice could carry part of the advantage, and Experiment 2 removed the difference. Both copies of a pair trained on the same answers, fixed in advance, in the same order, and received the same number of rewards. Only one copy's rewards followed the answers, and that copy again learned a new task faster. The advantage of the copies with control over their yoked twins could be forecast before the new task began from each copy's weights. In sum, control over rewards during earlier training left models faster at learning a new task later on, and a conventional model evaluation need not show it.

cs.LG↗

arXiv2026

ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on

Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should therefore be audited for reliance on concepts such as acquisition artifacts and patient demographics. Current methods, including linear probes and concept activation vectors, measure concept reliance by asking whether each concept is decodable from a layer. We show that this framing is misleading: decodability reflects not only concept reliance but also correlations among concepts and the outcome in the auditing dataset. We reframe shortcut auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON). ICON quantifies the share of a layer's variance explained by each concept conditional on all other concepts and the outcome, and also reports the variance left unexplained. Because ICON explanations are comparable across layers, their trajectories distinguish concepts that drive predictions from those that are merely encoded in the representations. In simulated data with ground truth, ICON recovers concept importance more accurately than seven baselines. In skin-cancer and neuroimaging applications, it separates learned shortcuts from merely correlated concepts, which we confirm with independent validation checks.

cs.LG↗

arXiv2026

Attributing Cohen's d: Training Data Attribution for Disease-Related Effects in Normative Age Biomarkers

Normative age models are trained to predict chronological age in a nominally healthy cohort. Applied to patients, they deviate, and the gap between predicted and chronological age is read as disease risk. Here, we attribute the disease-related effect size of the age gap directly to individual training samples, rather than using a prediction-level loss as the attribution target. For Cohen's $d$, the resulting closed-form influence functional, validated against leave-one-out retraining, ranks training samples by their effect on held-out case-control separation. Across four diseases and two biomarker modalities in UK Biobank, removing the 10% most influential training samples raises held-out disease-related effect size in every seed. It more than doubles the metabolomic-age effect for type-2 diabetes and raises the brain-age effect for multiple sclerosis by roughly a third. Random removal leaves effect size flat even at 50% removal, confirming the gain comes from which samples are removed, not how many. Flagged subjects carry subclinical cardiometabolic burden that diagnosis-based exclusion misses, on markers the model never sees. For type-2 diabetes, where the method gains most, the marker recovered is HbA1c, the standard measure of blood sugar control. We release pyinfluence, our influence-function package, for reproducibility and reuse.

cs.LG↗

arXiv2026

Measurement noise limits the advantage of nonlinear models over linear models in biomedical prediction

On biomedical tabular data, flexible models such as deep networks, gradient-boosted trees, and kernel methods are repeatedly matched or beaten by linear and logistic regression given the same features. The usual reaction is to treat this as a model-side shortfall, to be fixed with more data, a better architecture, or tuning, on the assumption that the nonlinear structure is there and the model has failed to capture it. We argue that these fixes cannot help when the binding limit is the measurement rather than the model, as it frequently is in biomedicine. Additive noise blurs the population-optimal predictor, and because blurring removes a function's fine, rapidly varying detail before its broad shape, it erases nonlinear structure faster than linear structure. A degree-$k$ interaction is attenuated by the $k$-th power of feature reliability, while the linear part is attenuated only once. At the reliabilities typical of biomedical measurement, the nonlinear advantage can vanish even when the underlying biology is strongly nonlinear, and what the noise removes cannot be recovered by a larger cohort or a more flexible model, only by better measurement. The nonlinearity is hidden, not absent, and a tie between linear and flexible models is not by itself a verdict on the biology. These pieces are classical, drawn from measurement-error statistics, psychometrics, and Gaussian analysis, and we assemble them into an exact excess-risk identity. Measurement reliability is one of three conditions, alongside sample size and feature representation, that must align for a flexible model to help, and together they leave only a narrow window that most biomedical tasks fall outside. Across 140 UK Biobank tasks, the gap between flexible and linear models, where it exists, carries the predicted noise signature, and the three conditions can be separated by intervention but not by a benchmark alone.

cs.LG↗

arXiv2026

Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Model

The development of foundation models for functional magnetic resonance imaging (fMRI) time series holds significant promise for predicting phenotypes related to disease and cognition. Current models, however, are often trained using a mask-and-reconstruct objective on small brain regions. This focus on low-level information leads to representations that are sensitive to noise and temporal fluctuations, necessitating extensive fine-tuning for downstream tasks. We introduce Brain-Semantoks, a self-supervised framework designed specifically to learn abstract representations of brain dynamics. Its architecture is built on two core innovations: a semantic tokenizer that aggregates noisy regional signals into robust tokens representing functional networks, and a self-distillation objective that enforces representational stability across time. We show that this objective is stabilized through a novel training curriculum, ensuring the model robustly learns meaningful features from low signal-to-noise time series. We demonstrate that learned representations enable strong performance on a variety of downstream tasks even when only using a linear probe. Furthermore, we provide comprehensive scaling analyses indicating more unlabeled data reliably results in out-of-distribution performance gains without domain adaptation.

cs.LG↗

arXiv2021

Label scarcity in biomedicine: Data-rich latent factor discovery enhances phenotype prediction

High-quality data accumulation is now becoming ubiquitous in the health domain. There is increasing opportunity to exploit rich data from normal subjects to improve supervised estimators in specific diseases with notorious data scarcity. We demonstrate that low-dimensional embedding spaces can be derived from the UK Biobank population dataset and used to enhance data-scarce prediction of health indicators, lifestyle and demographic characteristics. Phenotype predictions facilitated by Variational Autoencoder manifolds typically scaled better with increasing unlabeled data than dimensionality reduction by PCA or Isomap. Performances gains from semisupervison approaches will probably become an important ingredient for various medical data science applications.

cs.LG↗

arXiv2020

Clusters in Explanation Space: Inferring disease subtypes from model explanations

Identification of disease subtypes and corresponding biomarkers can substantially improve clinical diagnosis and treatment selection. Discovering these subtypes in noisy, high dimensional biomedical data is often impossible for humans and challenging for machines. We introduce a new approach to facilitate the discovery of disease subtypes: Instead of analyzing the original data, we train a diagnostic classifier (healthy vs. diseased) and extract instance-wise explanations for the classifier's decisions. The distribution of instances in the explanation space of our diagnostic classifier amplifies the different reasons for belonging to the same class - resulting in a representation that is uniquely useful for discovering latent subtypes. We compare our ability to recover subtypes via cluster analysis on model explanations to classical cluster analysis on the original data. In multiple datasets with known ground-truth subclasses, most compellingly on UK Biobank brain imaging data and transcriptome data from the Cancer Genome Atlas, we show that cluster analysis on model explanations substantially outperforms the classical approach. While we believe clustering in explanation space to be particularly valuable for inferring disease subtypes, the method is more general and applicable to any kind of sub-type identification.

q-bio.QM↗

arXiv2019

EMAP: Explanation by Minimal Adversarial Perturbation

Modern instance-based model-agnostic explanation methods (LIME, SHAP, L2X) are of great use in data-heavy industries for model diagnostics, and for end-user explanations. These methods generally return either a weighting or subset of input features as an explanation of the classification of an instance. An alternative literature argues instead that counterfactual instances provide a more useable characterisation of a black box classifier's decisions. We present EMAP, a neural network based approach which returns as Explanation the Minimal Adversarial Perturbation to an instance required to cause the underlying black box model to missclassify. We show that this approach combines the two paradigms, recovering the output of feature-weighting methods in continuous feature spaces, whilst also indicating the direction in which the nearest counterfactuals can be found. Our method also provides an implicit confidence estimate in its own explanations, adding a clarity to model diagnostics other methods lack. Additionally, EMAP improves upon the speed of sampling-based methods such as LIME by an order of magnitude, allowing for model explanations in time-critical applications, or at the dataset level, where sampling-based methods are infeasible. We extend our approach to categorical features using a partitioned Gumbel layer, and demonstrate its efficacy on several standard datasets.

cs.LG↗