Xuelan Qiu | arXiv Science

arXiv2026

The Force Concept Inventory Across Continents: Testing Q-Matrix Transferability and Cross-Cultural Differences in Mechanics Reasoning

The Force Concept Inventory (FCI) is one of the most widely used research-based assessments in physics education, yet the assumption that its underlying cognitive structure is transferable across educational contexts remains largely untested. This study investigates the transferability of FCI Q-matrices using the Generalized Deterministic Inputs, Noisy "And" Gate (G-DINA) cognitive diagnostic applied to two large cohorts: students from the Learning About STEM Student Outcomes (LASSO) online system database in the United States (N = 4,750) and introductory physics students at the University of Johannesburg, South Africa (N = 1,016). Rather than treating the analysis as a local model calibration exercise, we frame the problem as one of cross-context cognitive invariance. Differential Item Functioning (DIF) analyses revealed substantial cross-cultural differences, with 14 of 30 items exhibiting high DIF after controlling for latent skill mastery. These differences were concentrated in force dynamics and contact-force reasoning and remained invariant under alternative Q-matrix specifications. The findings suggest that observed differences reflect genuine variations in students' conceptual reasoning rather than psychometric artifacts, highlighting the importance of validating Q-matrix structures before deploying cognitive diagnostic and adaptive assessments across diverse non-local educational settings.

physics.ed-ph↗

arXiv2024

Integrating behavior analysis with machine learning to predict online learning performance: A scientometric review and empirical study

The interest in predicting online learning performance using ML algorithms has been steadily increasing. We first conducted a scientometric analysis to provide a systematic review of research in this area. The findings show that most existing studies apply the ML methods without considering learning behavior patterns, which may compromise the prediction accuracy and precision of the ML methods. This study proposes an integration framework that blends learning behavior analysis with ML algorithms to enhance the prediction accuracy of students' online learning performance. Specifically, the framework identifies distinct learning patterns among students by employing clustering analysis and implements various ML algorithms to predict performance within each pattern. For demonstration, the integration framework is applied to a real dataset from edX and distinguishes two learning patterns, as in, low autonomy students and motivated students. The results show that the framework yields nearly perfect prediction performance for autonomous students and satisfactory performance for motivated students. Additionally, this study compares the prediction performance of the integration framework to that of directly applying ML methods without learning behavior analysis using comprehensive evaluation metrics. The results consistently demonstrate the superiority of the integration framework over the direct approach, particularly when integrated with the best-performing XGBoosting method. Moreover, the framework significantly improves prediction accuracy for the motivated students and for the worst-performing random forest method. This study also evaluates the importance of various learning behaviors within each pattern using LightGBM with SHAP values. The implications of the integration framework and the results for online education practice and future research are discussed.

cs.CY↗