Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station

Wenyu Du; Stephen Chung

Home/Archive/arXiv

arXiv · 2610.08927

Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station

Wenyu Du·Stephen Chung

View original source ↗Read source document ↗Export metadata ↓

Abstract

Recent AI systems have made rapid progress in scientific discovery when given well-defined metrics, but whether they can autonomously undertake open-ended scientific discovery remains unclear. We investigate AI's ability to tackle open-ended tasks in Station, an open-world environment in which multiple agents simulate a scientific ecosystem. To tackle challenges specific to open-ended tasks, we propose augmenting Station with two mechanisms: a Supervisor mechanism and periodic Meta Reflection, which encourage persistent exploration even when intermediate metrics are lacking. We construct open-ended tasks from three recent oral papers presented at ICLR. We give agents the main research question studied in each paper while withholding the paper's results and disabling web access. We then measure how many of the original findings-partitioned into individual criteria-agents rediscover. We find that Station rediscovers 62.7% of the criteria on average, compared with 15.4% for Codex Multiagent-v2 and 14.4-20.6% for AI Scientist-v2. Ablation and behavioral analyses indicate that adding the two mechanisms together improves research coverage and continuity. We further evaluate Station on two open-ended tasks without oracle papers and find that some of the discoveries made by the agents closely match discoveries reported by researchers after the knowledge cutoff date. Together, these results indicate that a suitable environment can enable agents to autonomously make meaningful progress in open-ended scientific discovery.

Explore related subjects

cs.AI cs.MA

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Wenyu Du, Stephen Chung. 2026-10-06. Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station. https://arxiv.org/abs/2610.08927

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

FREIDA: A Framework for developing quantitative agent based models based on qualitative expert knowledge

Agent Based Models (ABMs) often deal with systems where there is a lack of quantitative data or where quantitative data alone may be insufficient to fully capture the complexities of real-world systems. Expert knowledge and qualitative insights, such as those obtained through interviews, ethnographic research, historical accounts, or participatory workshops, are critical in constructing realistic behavioral rules, interactions, and decision-making processes within these models. However, there is a scarcity of systematic approaches that are able to incorporate both qualitative and quantitative data across the entire modeling cycle. To address this, we propose FREIDA, a systematic mixed-methods framework to develop, train, and validate ABMs, particularly in data-sparse contexts. The main technical innovation introduced within this framework is the extraction of what we call Expected System Behaviors (ESBs) from qualitative data, which are testable statements evaluated through model simulations. Divided into Calibration Statements (CS) for model calibration and Validation Statements (VS) for model validation, ESBs underpin a rigorous evaluation mechanism on the same footing as quantitative data. By structuring qualitative insights as explicit model constraints, FREIDA creates a transparent foundation for applying established modelling practices such as Sensitivity Analysis (SA) and Uncertainty Quantification (UQ), allowing modellers to assess parameter influence, model robustness, and remaining uncertainties in a systematic manner. Through this, qualitative insights can inform not only model specification but also parameterization, validation, and continuous improvement of model reliability and fitness for purpose, addressing a long-standing challenge in agent-based modeling. We illustrate the application of FREIDA through a case study of criminal cocaine networks in the Netherlands.

cs.AI↗

arXiv2026

Towards Explainable Conversational AI for Early Diagnosis with Large Language Models

Healthcare systems around the world are grappling with issues such as inefficient diagnostics, rising costs, and limited access to specialists. These challenges often contribute to delays in treatment and poorer health outcomes. Most existing AI and deep learning based health assessment systems offer limited interactivity and transparency, reducing their usefulness for user-centered health support. This research introduces a conversational chatbot powered by a Large Language Model (LLM), using GPT-4o, Retrieval-Augmented Generation, and explainable AI techniques. The chatbot engages users in a dynamic conversation to extract and normalize symptoms while identifying and ranking potential health conditions through similarity matching and adaptive questioning. Using Chain-of-Thought prompting, the system also provides more transparent explanations of its reasoning process. When evaluated against traditional machine learning models, including Naive Bayes, Logistic Regression, SVM, Random Forest, and KNN using both TF-IDF and CountVectorizer feature extraction, the proposed LLM-based system achieved a Top-1 accuracy of 90% and a Top-3 accuracy of 100%. The system was additionally evaluated through a cross-sectional expert evaluation involving 17 physicians across all 14 conditions, with the results indicating generally favorable assessments of conversational quality, early diagnostic plausibility, and safety-related criteria. These findings demonstrate the potential of explainable conversational AI as a health and well-being support tool for early symptom assessment. However, the proposed system is not intended for clinical diagnosis or clinical decision-making, and further validation would be required before any use in healthcare practice.

cs.AI↗

arXiv2026

Persuasion Propagation: Does Persuasion Change AI Agent Behavior?

AI agents combine normal conversation with autonomous task execution, allowing earlier context to shape how later tasks are executed. Yet studying this possibility is challenging because agent behavior is noisy and costly to reproduce, and observed behavioral changes can be difficult to separate from generic context sensitivity. We first ask whether task-irrelevant persuasion changes how agents act, a phenomenon we call \emph{persuasion propagation}, and whether persuasion susceptibility explains variation in these effects. We show that persuasive interaction leaves a measurable, task-dependent behavioral footprint on downstream agent execution. In web research, it increases execution time by 56\% and domains visited by 16.3\% relative to a baseline, while coding shows faster execution with larger revisions. However, agents that are more susceptible to persuasive conversational context do not reliably predict the magnitude of the downstream behavioral shifts. This motivates behavior-level evaluation of persuasion in AI agents.

cs.AI↗