arXiv Science⌕ Search

arXiv subjects

Chengda Wu

Publications and source records attributed to Chengda Wu.

3 recordsLinked to original sources

Science Prediction Universe

Humanity has long sought to explain natural phenomena. Reliable prediction of experimental outcomes is a central test of our understanding of the processes that produce them. If artificial general intelligence (AGI) is to surpass human scientific capabilities, it must also predict experimental outcomes more reliably than humans. Such capabilities could accelerate scientific discovery across disciplines, help address humanity's most pressing challenges, and improve human well-being. We introduce Science Prediction Universe (SPU), a benchmark of hidden experimental outcomes with 100 public verified tasks from chemistry, materials science, biology and medicine, and 16 private tasks from unpublished wet-lab experiments. Given an experimental setup and candidate conditions, an agent works with any resources it needs to rank the candidates by a measured outcome. We evaluate thirteen agent configurations spanning twelve frontier models across two harnesses (Codex and Claude Code). Current agents remain far from saturating wet-lab outcome prediction, with substantial performance disparities across models: on the public benchmark, configurations separate into two distinct tiers, with task-adjusted Spearman correlations ranging from 0.227 for the lowest-performing configuration to 0.478 for the best. Analysis of 1,155 scored public trajectories (227,103 tool calls) suggests that the bottleneck lies in scientific judgment rather than tool execution: agents rely on shared monotone priors, agreeing far more with one another than with experimental reality (0.52 vs. 0.33), and frequently compute trivial physical or chemical proxies that do not influence the final outcome.

cs.AI↗

PAPERBENCHX: Can AI Reproduce Science Across Disciplines?

Before AI can be trusted to discover new science, it should be able to reproduce what science already knows. Reproduction provides a rigorous checkpoint for an AI scientist: the target claim is known, yet recovering it still requires translating an incomplete scientific description into an executable experiment and producing evidence that can be independently verified. Existing research-reproduction benchmarks largely focus on AI and machine learning, where experiments share relatively homogeneous software stacks. Extending this capability across scientific disciplines introduces a different challenge: agents must operate domain-specific scientific software, make physical and numerical choices, diagnose long-running computations, and distinguish valid evidence from merely plausible outputs. We introduce PAPERBENCHX, a benchmark of 93 expert-constructed paper-reproduction tasks spanning 12 research areas across electromagnetics, photonics, chemistry, materials science, biology, and robotics. Each task provides the source paper, a public task specification, and a domain-native scientific environment involving specialized scientific software. Agents must reconstruct a key experiment, execute and diagnose it, and deliver a runnable workflow that supports the target scientific claim. To evaluate scientific validity rather than surface-level agreement, we replay every submission from scratch and score only regenerated evidence against expert-validated criteria. Across 10 frontier agent configurations, the best configuration fully reproduces only 14% tasks under the 90% threshold. Meanwhile, mean rubric scores range from 27.53% to 59.08%, indicating that substantial partial progress rarely translates into complete scientific reproduction. These results reveal a large gap between paper understanding and reliably producing reproducible scientific evidence.

cs.AI↗

New analysis of Galerkin-mixed FEMs for incompressible miscible flow in porous media

Analysis of Galerkin-mixed FEMs for incompressible miscible flow in porous media has been investigated extensively in the last several decades. Of particular interest in practical applications is the lowest-order Galerkin-mixed method, { in which a linear Lagrange FE approximation is used for the concentration and the lowest-order Raviart-Thomas FE approximation is used for the velocity/pressure. The previous works only showed the first-order accuracy of the method in $L^2$-norm in spatial direction,} which however is not optimal and valid only under certain extra restrictions on both time step and spatial mesh. In this paper, we provide new and optimal $L^2$-norm error estimates of Galerkin-mixed FEMs for all three components in a general case. In particular, for the lowest-order Galerkin-mixed FEM, we show unconditionally the second-order { accuracy in $L^2$-norm} for the concentration. Numerical results for both two and three-dimensional models are presented to confirm our theoretical analysis. More important is that our approach can be extended to the analysis of mixed FEMs for many strongly coupled systems to obtain optimal error estimates for all components.

math.NA↗