arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2508.13201

Benchmarking LLM-based agents for single-cell omics analysis

Yang Liu·Lu Zhou·Xiawei Du·Ruikun He·Xuguang Zhang·Rongbo Shen·Yixue Li
View original source ↗Read source document ↗Export metadata ↓

Abstract

Background: The surge in single-cell omics data exposes limitations in traditional, manually defined analysis workflows. AI agents offer a paradigm shift, enabling adaptive planning, executable code generation, traceable decisions, and real-time knowledge fusion. However, the lack of a comprehensive benchmark critically hinders progress. Results: We introduce a novel benchmarking evaluation system to rigorously assess agent capabilities in single-cell omics analysis. This system comprises: a unified platform compatible with diverse agent frameworks and LLMs; multidimensional metrics assessing cognitive program synthesis, collaboration, execution efficiency, bioinformatics knowledge integration, and task completion quality; and 50 diverse real-world single-cell omics analysis tasks spanning multi-omics, species, and sequencing technologies. Our evaluation reveals that Grok3-beta achieves state-of-the-art performance among tested agent frameworks. Multi-agent frameworks significantly enhance collaboration and execution efficiency over single-agent approaches through specialized role division. Attribution analyses of agent capabilities identify that high-quality code generation is crucial for task success, and self-reflection has the most significant overall impact, followed by retrieval-augmented generation (RAG) and planning. Conclusions: This work highlights persistent challenges in code generation, long-context handling, and context-aware knowledge retrieval, providing a critical empirical foundation and best practices for developing robust AI agents in computational biology.

Explore related subjects

q-bio.GNcs.AIcs.MA

Keep this discovery

BibTeXRIS

Yang Liu, Lu Zhou, Xiawei Du, Ruikun He, Xuguang Zhang, Rongbo Shen, Yixue Li. 2025-08-16. Benchmarking LLM-based agents for single-cell omics analysis. https://doi.org/10.1186/s13059-026-03998-z

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Transformer-Based Delta Expression Encoder for Psilocybin Transcriptional Response: Architecture, Representations, and Biological Validation

Understanding why individuals respond differently to psilocybin requires modeling the drug's transcriptional perturbation signature at the cell-type level. I present a Transformer-based delta expression encoder that learns to classify differential gene expression status - upregulated, downregulated, or neutral - from single-nucleus RNA-sequencing data, without supervision from pathway annotations or prior biological knowledge. The model is trained on pseudobulk profiles from 623 examples spanning 18 cell types, 2 drug conditions, and 6 timepoints derived from the Liao et al. 2025 dataset, and achieves 69.4% weighted classification accuracy. Three principal findings are reported, alongside one direct test of a published hypothesis that returned a result inconsistent with that hypothesis. First, per-cell-type classification accuracy ranges from 28.3% (L2/3 IT, a primary HTR2A-expressing psilocybin target) to 99.6% (endothelial cells), consistent with known psilocybin response biology. Second, psilocybin-induced transcriptional downregulation is significantly more stereotyped across individuals than upregulation (Mann-Whitney U=18615.0, p<0.0001), a novel finding with a cortical depth gradient across excitatory subtypes. Third, attention-guided gene co-regulation analysis recovers drug-specific modules without pathway supervision. Separately, a direct test of whether baseline HTR2A expression predicts drug-response separability across cell types found a significant negative correlation (Spearman r = -0.7088, p = 0.0021), the opposite of what a simple HTR2A-gating account would predict.

q-bio.GN↗
arXiv2026

Human mutation field reveals an equilibrium-like structure with irreversible circulation

The evolution of DNA sequences can be viewed as stochastic dynamics on a high-dimensional discrete space, but it is unclear when empirical transition biases reduce to an effective energy landscape versus retain irreducible non-equilibrium circulation. Human context-dependent mutation probabilities offer a direct test: every single-nucleotide substitution in a local context has a reverse substitution, so the logarithm of the forward-to-reverse probability ratio defines an antisymmetric field-the human mutation field. We show this field has a dominant gradient component and a smaller but reproducible curl component. Using seven-base human germline substitution probabilities, we infer an effective mutational landscape with a Siamese neural network constrained to predict only energy differences. This model predicts forward-to-reverse log-ratios for held-out mutations with a correlation of about 0.93, close to both an unconstrained predictive reference (0.948) and the empirical reversible ceiling from Hodge projection (about 0.96). Although trained only on mutation probabilities, the inferred landscape largely recovers short-word genomic composition and Chargaff reverse-complement symmetry for sequences up to length four. Deviations from equilibrium structure reveal a small but detectable nonequilibrium component: a residual irreversible circulation violating the Kolmogorov cycle condition for detailed balance, reproducible across African, Asian, and European populations, and strongest in CpG-linked cycles and CpG-transition edges, consistent with methylcytosine deamination. These results give a thermodynamic decomposition of the human mutation field: most mutation bias is organized by a local equilibrium-like energy landscape aligned with genome composition, while the residual circulation points to specific directional mutational mechanisms.

q-bio.GN↗
arXiv2026

Subcellularly Resolved Single-Cell Embedding Learning with Transcriptomic data, Protein Structure and Localization Information

Existing cell embedding methods predominantly rely on transcriptomic or proteomic measurements and represent each cell as a holistic entity, thereby overlooking the subcellular localization of individual molecules. Moreover, they rarely incorporate protein structural information, despite its fundamental role in determining molecular interactions and functions. In this work, we propose a multimodal framework for learning subcellularly resolved cell embeddings by jointly leveraging RNA expression profiles, protein sequence representations, and protein structural information. Specifically, we employ a cross-attention architecture to integrate transcriptomic, sequence, and structural modalities and model their interactions within distinct subcellular compartments. The resulting embeddings represent each cell through its fine-grained subcellular organization, capturing both molecular expression patterns and the functional properties of the associated proteins. By learning cell representations at subcellular resolution, our framework preserves spatially organized biological information while integrating complementary signals across multiple molecular levels. To the best of our knowledge, this is the first framework that produces subcellularly resolved cell embeddings by jointly incorporating transcriptomic information, protein sequence representations, and protein structural knowledge within a unified cross-modal learning paradigm.

q-bio.GN↗