arXiv Science⌕ Search

arXiv · 2611.14184

Benchmarking large language models for parameter-level structural identifiability classification in epidemic ODE models

Abstract

Structural identifiability determines whether unknown parameters can be uniquely recovered from ideal model outputs, making it a prerequisite for interpreting parameter estimates. We evaluated whether large language models (LLMs) can reliably provide preliminary structural identifiability classifications for epidemic models. We constructed a retrospective benchmark of 88 scenarios from 14 of the 27 studies considered, encompassing 696 parameter-level formal reference classifications from published results or Julia-based symbolic analysis. Scenarios were evaluated using Domain-Aware and canonically relabeled ("Domain-Anonymized") prompts, with ten runs per configuration and prompt condition. Six configurations completed all scheduled responses, yielding 83,520 parameter-level decisions for the primary comparison. GPT-5.5 Instant and GPT-5.5 Thinking Medium had the highest accuracies among complete configurations (88.71% and 88.07%). Canonical relabeling improved both configurations and attenuated their performance decline in scenarios with many unknown parameters or state variables, although effects differed across model families. Locally identifiable parameters were the most difficult class (highest strict recall among complete configurations: 42.00%), although the local reference set was small. In multivariable analysis, each additional unknown parameter reduced the odds of correct classification (OR 0.881; 95% CI 0.852-0.910), whereas additional observed outputs increased them (OR 1.666; 95% CI 1.503-1.846). Exploratory evaluations indicated strong performance for GPT-5.5 Thinking High and Claude Opus 5, despite incomplete coverage. LLMs may generate preliminary hypotheses or flag cases for formal review, provided outputs are checked against published results or formal symbolic methods. They should not be used to certify the structural identifiability of a complete model.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Geunsoo Jang, K. Selçuk Candan, Gerardo Chowell. 2026-11-10. Benchmarking large language models for parameter-level structural identifiability classification in epidemic ODE models. https://arxiv.science/abs/2611.14184

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Optimising Recruitment and Sample Allocation in Multi-Site Rare Disease Studies

Clinical research in rare diseases is challenged by the limited availability and broad geographical distribution of eligible patients. This necessitates multi-centre collaboration and strategic selection of analytical laboratories based on capacity, cost, and capability. We present a mathematical framework based on cardinality optimisation to support key operational decisions in study design, balancing clinical relevance, feasibility, budgetary constraints, and logistical factors. The framework was implemented in the Reconstruction and Computational Modelling for Inherited Metabolic Diseases (Recon4IMD) project, aiming to recruit 1,900 diagnosed patients across 24 healthcare centres and eight metabolomic laboratories. Compared to both proportional heuristic and greedy marginal allocation baselines, the proposed optimisation model consistently achieved higher recruitment yields in simulated multi-centre scenarios and reduced metabolomic analysis costs by approximately 30% under unconstrained conditions. Notably, both the heuristic and greedy allocation baselines became infeasible under progressively tighter laboratory capacity constraints, whereas the optimised model maintained feasibility and cost control. These findings highlight the value of integrated optimisation strategies for improving recruitment, platform selection, and sample allocation in complex, resource-limited research settings.

q-bio.QM↗

Analysis choices can produce apparent rich-club differences: an Alzheimer's disease EEG case study

Background: The weighted rich-club coefficient tests whether a network's most strongly connected nodes are preferentially interconnected. Its interpretation depends on two often-implicit choices: how the randomised comparison networks are constructed, and whether the hubs are re-identified in each comparison network or held fixed at those of the observed network. When hubs are defined by connection strength, these choices change what the coefficient tests. New method: We examined how these choices affect inference using resting-state EEG from 20 people with Alzheimer's disease and 20 controls. Holding the data fixed, we compared four analysis pipelines, crossing two ways of building the comparison networks (global weight reshuffle or strength-conditioned null) with the two hub choices, and ran a calibration study using data constructed to contain no preferential hub coupling beyond nodal strength. Results: An apparent eyes-open group difference in rich-club organisation appeared only under the pipeline combining a global weight reshuffle with re-identified hubs. Under the empirical resampling ensemble, that pipeline reported a group difference in 43% of pseudo-studies, always higher in the Alzheimer's disease group, while the other three stayed near the expected 5%. Comparison with existing methods: Unlike global weight reshuffling, strength-conditioned nulls approximately preserve nodal strengths; fixing observed hubs also retains the identity of the core being tested. Conclusions: The same data can support or dissolve an apparent rich-club difference depending on these choices. No single choice is universally correct, but reporting which were made, and why they fit the question being asked, is essential to interpreting the result.

q-bio.QM↗

Quantum Cognition Machine Learning for Forecasting Chromosomal Instability

The accurate prediction of chromosomal instability from the morphology of circulating tumor cells (CTCs) enables real-time detection of CTCs with high metastatic potential in the context of liquid biopsy diagnostics. However, it presents a significant challenge due to the high dimensionality and complexity of single-cell digital pathology data. Here, we introduce the application of Quantum Cognition Machine Learning (QCML), a quantum-inspired computational framework, to estimate morphology-predicted chromosomal instability in CTCs from patients with metastatic breast cancer. QCML leverages quantum mechanical principles to represent data as state vectors in a Hilbert space, enabling context-aware feature modeling, dimensionality reduction, and enhanced generalization without requiring curated feature selection. QCML outperforms conventional machine learning methods when tested on out of sample verification CTCs, achieving higher accuracy in identifying predicted large-scale state transitions (pLST) status from CTC-derived morphology features. These preliminary findings support the application of QCML as a novel machine learning tool with superior performance in high-dimensional, low-sample-size biomedical contexts. QCML enables the simulation of cognition-like learning for the identification of biologically meaningful prediction of chromosomal instability from CTC morphology, offering a novel tool for CTC classification in liquid biopsy.

q-bio.QM↗