arXiv Science⌕ Search

arXiv · 2611.13404

AI is ready, but are the students. From answer-making to sense-making

Abstract

Students in introductory physics often struggle with interpreting kinematics graphs, especially when connecting graphical features to physical meaning. We investigated AI tool use in kinematic graphing analysis as an intervention to support active sensemaking. This study examines GPT-5 performance on the Test of Understanding Graphs in Kinematics (TUG-K) and the way students interacted with GPT-5 during introductory physics activities involving graphical analysis. For the first part of the study, GPT-5 responses were evaluated across repeated TUG-K trials and compared to previously reported GPT-4 performance. Following that analysis, student-GPT-5 dialogue episodes were collected and analyzed using an inductive coding scheme. GPT-5 performance results show markedly improved accuracy. Student-GPT-5 dialogue episodes evolved from mostly procedural answer-making requests early in the term, to more reflective sensemaking interactions at the end of the term, suggesting that instructional framing is needed for AI tools to function as meaningful peer-like learning supports.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Antoinette Stone, Ruoning Zhang, Brian Utter. 2026-11-10. AI is ready, but are the students. From answer-making to sense-making. https://arxiv.science/abs/2611.13404

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

FlowMLLab: An open-source framework for reproducible computational-fluid-dynamics and scientific-machine-learning experiments

FlowMLLab is an open-source Python framework for scientific machine learning in thermal-fluid and aerospace engineering, covering incompressible, compressible and rarefied flows. Its 22 weeks comprise 51 notebooks and 35 lectures. Solvers and molecular sampling underpin field prediction, uncertainty quantification and reduced-order modeling. Proper orthogonal decomposition (POD) and deep operator networks (DeepONets), including POD-DeepONet, connect operating conditions to flow fields alongside Fourier neural operators. Physics-informed neural networks infer solutions and constitutive coefficients. Geometry-conditioned models, including the point-cloud DeepONet variant Geom-DeepONet, handle changing domains. Aerospace applications address airfoil shock/vortex identification, rarefied-gas modeling, nozzle flow prediction and supersonic body design. Complementary thermal-fluid examples cover unsteady wakes, hydrofoil vapor clouds and noisy particle observations. Attention mechanisms support sparse-sensor reconstruction, autoregressive prediction and transfer assessment, alongside character-level language modeling. Complete-case evaluation excludes tests from selection; execution notes identify retained research outputs. A worked extension embeds a learned monotone relation for a fixed manufactured Carreau fluid in channel momentum balance. At three unseen in-support pressure gradients, it outperforms profile-interpolation and fitted-Newtonian controls with different information or priors.

physics.ed-ph↗

Stress Testing a Multimodal Large Language Model Grading Assistant on Kinematics Graph Reading

Multimodal large language models (MLLMs) can recognize graph features yet award credit without sufficient graphical support. This exploratory study examines selected grading outputs for synthetic responses to four kinematics exercises, generated through visual distortion and progressive transformation toward reference graphs. Different model and prompt configurations were used. Some judgments correctly distinguished instantaneous zero velocity from rest over a finite interval and recognized positive curvature. Yet a severely distorted velocity--time graph received full credit despite disconnected and branching traces and an unclear zero-velocity reference. Other submissions graded under the same configuration lost substantial credit. Numerical cutoffs intended to distinguish meaningful features from drawing irregularities were supplied without a coordinate convention for unnumbered graphs, leaving the basis of exact partial-credit awards unresolved. Stress testing can expose unsupported awards before classroom use. For the configurations examined, we do not recommend accepting scores as final grades without instructor review, including review of full-credit responses.

physics.ed-ph↗

Students' Validation Criteria for AI-Generated Physics Solutions

A physics solution can be clear without being valid. This distinction becomes important when students read AI-generated solutions that appear coherent but still require physical and mathematical checking. We examined how pre-service physics teachers in a calculus-based introductory electromagnetism course evaluated AI-generated solutions to a charged-rod electric-field problem after attempting the problem themselves. Although the AI-generated solutions analyzed were correct, no student accepted a response as valid after a first reading. Students who had solved the problem fully or in part could examine the responses using physical and mathematical criteria, while others withheld acceptance because they could not follow the mathematical development. These results suggest that, in physics instruction, AI-generated solutions should be treated less as answers to be accepted or rejected and more as worked-out responses that students learn to reconstruct, question, and test.

physics.ed-ph↗