Dynamic Models of Learning and Education Measurement

Abstract

Pre-post testing is a commonly used method in physics education community for evaluating students' achievement and or the effectiveness of teaching through a specific period of instruction. A popular method to analyze pre-post testing results is the normalized gain first brought to the physics education community in wide use by R.R. Hake. This paper presents a measurement based probabilistic model of the dynamic process of learning that explains the experimentally observed features of the normalized gain. In Hake's study with thousands of students' pre-post testing results, he observed that on average 48 courses employing "interactive engagement" types of instruction achieved average normalized gains about two standard deviations greater than did 14 courses subjected to traditional instruction. For all courses the average normalized gains had a very low correlation +0.02 with average pretest scores. This feature of the normalized gain has allowed researchers to investigate the effectiveness of instruction using data collected from classes with widely different average pretest scores. However, the question of why the average normalized gain has this feature and to what extent this feature is generally present is not well understood. In addition, there have been debates as to what the normalized gain actually measures, and concerns that it lacks a probability framework that undergirds psychometric methods such as Item Response Theory (IRT). The present model leads to an explanation of the observed features of the normalized gain, connects to other models such as IRT, and shows that the normalized gain does have a probability framework but one different from that emphasized by IRT.