arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.36226

Structure, Not Sequence: Tracking and Validating Folding Mechanisms in ESMFold's Trunk with Sparse Autoencoders

Bohan Liu·Xiaohan Fan·Wenqian Ye·Aidong Zhang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Structure predictors like ESMFold fold proteins using neural network computation and have transformed structural biology. However, the driving mechanism from amino-acid (AA) sequence to 3D structure is insufficiently understood, and biological meaning remains locked in the dense network. Sparse autoencoders (SAEs) have shown promising results in obtaining interpretable concept features over an overcomplete basis of intermediate protein sequence-model representations. Yet folding models, which predict physical shape from AA sequence semantics, have no such account. We hypothesize that the features carrying the folding computation evolve in the 3D structural regression process. In this paper, we interpret the folding trunk in three steps. First, a pilot analysis on the representation distribution across modules and recycles locates distinct spatial computation in the trunk. Second, the dictionaries expose a division of labor: the single representation carries inherited semantics, and the pair representation computes the pairwise geometry. Third, we interpret the dictionary at scale with a reasoning language model, trace how the concepts compose across blocks by gradient-based attribution, and follow the interpreted code to the model's folding output, verifying where the computation aligns with, or departs from, known biochemistry. Together, the sparse dictionaries we train capture the relational geometry the trunk computes beyond the sequence model, and their concept activations forecast the model's own fold quality, turning the black-box folding trunk into a sparse, interpretable, and scientifically verifiable circuit.

Explore related subjects

q-bio.QMcs.AIcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Bohan Liu, Xiaohan Fan, Wenqian Ye, Aidong Zhang. 2026-10-26. Structure, Not Sequence: Tracking and Validating Folding Mechanisms in ESMFold's Trunk with Sparse Autoencoders. https://arxiv.org/abs/2610.36226

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multimodal Knowledge Aggregation for Microbial Keratitis: Geographic Confounding and Shortcut Risk in Pooled Multicountry Data

To develop an open-source multimodal model for bacterial-versus-fungal microbial keratitis classification and assess shortcuts from geographic pooling. Participants: 1,645 US and Indian patients contributed 17,158 slit-lamp images under blue-light, sclerotic-scatter, and white-light illumination. Methods: The model combined cross-modality contrastive learning, modality-specific fine-tuning, and fusion with 7 clinical variables. Diagnoses used smear, culture, and polymerase chain reaction results, with expert review of culture-negative cases. The model and 5 comparators used 5 patient-disjoint outer folds stratified by country and organism, with inner validation. Secondary analyses examined within-country performance, a country-only reference, ablations, and imbalance mitigation. Code: https://github.com/yqwang01/TPMKA. Main Outcome Measures: Patient-level macro-F1, class-specific F1, accuracy, and area under the receiver operating characteristic curve (AUROC), overall and by country. Results: The model had the highest overall macro-F1 (83.06%; 95% CI, 81.58-84.56) and accuracy (84.50%; 95% CI, 83.16-85.84) among matched comparators, with 89.9% AUROC (95% CI, 88.7-91.1). Paired differences were not consistently significant. AUROC was 83.9% in India and 67.0% in the US; bacterial F1 in India was 47.78%, and fungal F1 in the US was 40.97%. A country-only reference achieved 81.88% accuracy and 78.2% AUROC overall despite 50.0% AUROC within each country. Resampling and group distributionally robust optimization improved some underrepresented groups without removing the geographic pattern. Conclusions: The model compared favorably with matched methods, but geographic prevalence differences made pooled performance more reassuring than within-country discrimination. Independent geographic validation is needed.

q-bio.QM↗
arXiv2026

Genotypic Triggers: Exposing Pharmacogenomic Blind Spots via Host-Specific Backdoors in Generative Antimicrobial Peptide Models

Large Language Models (LLMs) have accelerated drug discovery, particularly in the automated design of antimicrobial peptides (AMPs). However, current validation pipelines for peptide generation models overlook historical precedents showing that certain drugs carry health risks predominantly for individuals with specific genetic profiles. In this paper, we demonstrate that such targeted health risks can be induced intentionally and at scale by manipulating models that generate peptide candidates. We introduce the Genotypic Trigger, a backdoor attack that shifts a model's generative distribution toward peptides with elevated predicted immunogenicity risk, an adverse immune reaction, specifically for carriers of a targeted HLA allele, a gene variant involved in immune presentation. Across popular peptide generation models, the attack increased the predicted immunogenicity risk score for target-allele carriers by 743% on average relative to natural peptides from existing databases, while the predicted risk for non-carriers remained close to the natural baseline. Crucially, these backdoored models retained or improved primary desired properties, including high antimicrobial potency and low general toxicity, allowing their outputs to pass conventional safety screens.

q-bio.QM↗
arXiv2026

Shared clones make random cell halves look reproducible in pooled screens

Single-cell perturbation screens are used to train and evaluate models of perturbation effects, and model scores are judged against the reproducibility of the measured effects. Reproducibility is usually judged by splitting each perturbation's cells into two random halves, assumed to be independent replicates. In pooled screens such as Perturb-seq, the cells carrying a guide RNA descend from a limited number of transduced cells, so two random halves contain the same clones: pseudoreplication whose unit is not marked in the data. Using null perturbations (no expected effect) as controls, we compared random halves with independent replicates from separate transductions, and tested directly with halves that differ only in their clones. In a genome-scale screen of primary T cells, null perturbations agreed between random halves almost always and between independent replicates only half the time, the rate expected from noise alone; more cells did not remove it. With halves that share no clone, split by the T-cell receptor constant region or by a clone barcode, the excess agreement largely disappeared. Under a simple model, consistent with a screen that recorded clones, its size equals the share of expression differences set by clones divided by the effective number of clones per perturbation; in cell lines an upper bound on it was small. In the T-cell screen, shared clones inflated benchmark ceilings and reliability labels, the more so in donors with fewer effective clones, and when a model's input and scoring answer come from one transduction, published models that predict effects in a new cell state scored well even on perturbations with no expected effect; for the model tested (STATE), the gain vanished with the same guide from another transduction as answer. Reproducibility is better judged across separate transductions than between random halves.

q-bio.QM↗