arXiv subjects

Steven J. Pollock

Publications and source records attributed to Steven J. Pollock.

At least 19 recordsLinked to original sources

arXiv2026

Assessing student learning in quantum computing: Lessons from developing a test item on phase kickback

A major challenge for quantum workforce development is the need to both understand and reliably assess student learning of quantum information science (QIS) fundamentals. Yet student thinking is notoriously difficult to probe, even for seasoned education researchers. This article presents the story of Item 15 on the Quantum Computing Conceptual Survey (QCCS). This assessment item underwent more revision and discussion within the team than the remaining 19 assessment questions combined. This paper provides a behind-the-scenes look at the development of this assessment question: a story that both reveals interesting findings about student reasoning in quantum computing and illustrates why quantum education researchers insist on triangulating diverse quantitative and qualitative data sources when developing and refining assessment items, with implications for any researcher looking to understand and measure student conceptual reasoning in quantum computing, as well as for QIS curriculum and workforce development more broadly.

physics.ed-ph↗

arXiv2026

Measuring student understanding in quantum computing: Development and validation of the Quantum Computing Conceptual Survey

Research-based assessments (RBAs) have proven to be valuable tools in PER, supporting both instructional reform and foundational research. In the rapidly-growing field of Quantum Information Science (QIS), the lack of suitable RBAs limits the field's ability to make evidence-based decisions about curricula and program development. In this paper, we introduce the Quantum Computing Conceptual Survey (QCCS), an instrument developed to help address this gap by measuring student conceptual understanding in the foundations of quantum computing. Using pilot data from over 50 courses and 700 students, we present evidence supporting the validity of the QCCS for use in introductory QIS courses, drawing on analyses from both classical test theory and the Rasch model. We detail the potential uses for the QCCS, intending for it to serve as both a practical tool for instructors and a means of facilitating measurement-driven QIS education research.

physics.ed-ph↗

arXiv2026

Computer-generated QIS tutorial feedback is valued by students, but does not replicate in-class collaboration

PER has consistently demonstrated the effectiveness of small-group tutorials in helping students develop conceptual understanding and fluency, but instructor uptake is limited by resource constraints. To test the effectiveness of out-of-class tutorials using computer-generated feedback as an instructor-friendly alternative, we conducted think-aloud interviews with students in a quantum computing course who were randomly assigned to either a traditional validated small-group, pencil-and-paper tutorial on tensor products, or a solo computerized adaptation thereof. We found that while the computer-generated feedback was broadly considered useful by students, student engagement patterns were markedly different in the solo setting, with students demonstrating reluctance to use the interface's built-in help features and tending to internalize failure in unproductive ways counter to our intention of a formative learning environment. We discuss implications for curriculum design and directions for future research that may help to answer the longstanding question in PER of why tutorials work so well.

physics.ed-ph↗

arXiv2026

Can a CNOT Gate Affect the Control Qubit? Student Resources for Understanding CNOT and Entanglement

The Controlled-Not (CNOT) gate is essential to algorithms in quantum computing for its ability to entangle qubits. As such, it is important to understand how students learning quantum computing reason around the function and use of this critical quantum gate. To investigate this, we conducted think-aloud interviews in which students solved problems involving the CNOT gate to understand students' `CNOT toolbox' -- the strategies and cognitive resources students use when reasoning about the effect of the CNOT gate. We identify three cognitive resources related to the CNOT gate: (1) the procedural resource of applying CNOT to specific states, (2) a qualitative description of CNOT's effect on the target qubit given the control qubit, and (3) the idea that the control qubit is not changed when CNOT is applied to computational basis states. We find that students' use of the first resource is foundational to their understanding of the second and third, that the second and third resources can sometimes lead students to incorrect conclusions, and that students can use each of these resources separately or in tandem. We also explore how students use these resources in conjunction with Dirac notation, superposition states, and entanglement to reason both productively and unproductively about quantum computing problems.

physics.ed-ph↗

arXiv2023

Introductory quantum information science coursework at US institutions: Content coverage

Despite rapid growth of quantum information science and engineering (QIS/QISE) workforce development initiatives, perceived lack of agreement among faculty on core content has made prior research-based curriculum and assessment development initiatives difficult to scale. To identify areas if consensus on content coverage, we report findings from a survey of N=63 instructors teaching introductory QISE courses at US institutions of higher learning. We identify a subset of content items common across a large fraction (>=80%) of introductory QISE courses that are potentially amenable to research-based curriculum development, with an emphasis on foundational skills in mathematics, physics, and engineering. As a further guide for curriculum development, we also examine differences in content coverage by level (undergraduate/graduate) and discipline. Finally, we briefly discuss the implications of our findings for the development of a research-based QISE assessment at the postsecondary level.

physics.ed-ph↗

arXiv2023

How media hype affects our physics teaching: A case study on quantum computing

Popular media is an unspoken yet ever-present element of the physics landscape and a tool we can utilize in our teaching. It is also well-understood that students enter the physics classroom with a host of conceptions learned from the world at large. It stands to reason, then, to suspect that media coverage may be a major contributing factor to students' views on physical phenomena and the nature of science - one whose influence will only grow amid the 21st century digital age. Yet the role of the media in shaping physics teaching and learning has remained largely unexplored in the physics education research (PER) literature so far. Here, we explore the phenomenon of media hype from a theoretical and practical perspective: how media rhetoric of current topics in science and technology evolves, and how it affects students and instructors. We argue that media hype of cutting-edge science can be a double-edged sword for educators, with the same amped-up rhetoric that motivates students to enter the classroom tending to result in inflated preconceptions of what the science and technology can actually do. We draw on examples related to teaching quantum computing as a case study, though the findings we present should generalize to other topics garnering significant media attention - from exoplanets to graphene to batteries for electric vehicles. We conclude with a set of practical recommendations for physics teachers at all levels who wish to be more cognizant of the role exposure to popular media has on students and to tailor our teaching accordingly.

physics.ed-ph↗

arXiv2022

Investigating student interpretations of the differences between classical and quantum computers: Are quantum computers just analog classical computers?

Significant attention in the PER community has been paid to student cognition and reasoning processes in undergraduate quantum mechanics. Until recently, however, these same topics have remained largely unexplored in the context of emerging interdisciplinary quantum information science (QIS) courses. We conducted exploratory interviews with 22 students in an upper-division quantum computing course at a large R1 university crosslisted in physics and computer science, as well as 6 graduate students in a similar graduate-level QIS course offered in physics. We classify and analyze students' responses to a pair of questions regarding the fundamental differences between classical and quantum computers. We specifically note two key themes of importance to educators: (1) when reasoning about computational power, students often struggled to distinguish between the relative effects of exponential and linear scaling, resulting in students frequently focusing on distinctions that are arguably better understood as analog-digital than classical-quantum, and (2) introducing the thought experiment of analog classical computers was a powerful tool for helping students develop a more expertlike perspective on the differences between classical and quantum computers.

physics.ed-ph↗

arXiv2022

The interdisciplinary quantum information classroom: Themes from a survey of quantum information science instructors

Interdisciplinary introduction to quantum information science (QIS) courses are proliferating at universities across the US, but the experiences of instructors in these courses have remained largely unexplored in the discipline-based education research (DBER) communities. Here, we address this gap by reporting on the findings of a survey of instructors teaching introduction to QIS courses at institutions across the US, primarily at the undergraduate or hybrid undergraduate/graduate level, as well as follow-up focus interviews with six individual instructors. Key themes from this analysis include challenges and opportunities associated with the diversity of instructor and student backgrounds, student difficulties with the mathematical formalism (especially though not exclusively with linear algebra), and the need for better textbooks and curricular materials. We also find that while course topics are ostensibly similar, each course is crafted by its instructor to tell a different story about QIS and to uniquely balance goals such as accessibility and academic rigor, such that no canonical introduction to QIS course emerges from our dataset. We discuss the implications of this finding with regard to the benefits and risks associated with standardization of curricula as QIS coursework matures.

physics.ed-ph↗

arXiv2020

Retention of conceptual learning after an interactive introductory physics course

The cyclic format of the undergraduate physics curriculum depends on students' ability to recall and utilize material covered in prior courses in order to reliably build on that knowledge in later courses. However, there is evidence to suggest that people often do not retain all, or even most, of what they learned previously. How much information is retained appears to be dependent both on the individuals' approach to learning as well as the style of instruction. In particular, there is evidence to suggest that active engagement techniques in the classroom can improve students' retention of the material over time. Here, we report the findings of a longitudinal investigation of students' retention of conceptual understanding as measured by the Force and Motion Conceptual Evaluation (FMCE) following a first-semester, calculus-based introductory physics course, which features significant active engagement in both lecture and recitation. By administering the FMCE at the end of a first-semester physics course and again at the beginning of the subsequent second-semester physics course, we examine students' knowledge retention over time periods ranging from 1-15 months. We find that the shift in students' FMCE scores between these two courses is positive but corresponds to a small effect size, indicating that students retained effectively all of their conceptual learning (as measured by the FMCE). This finding largely persists even as the length of the gap between the two courses increases. We also find that, when breaking out students' performance on individual questions, the majority of students maintain their score on individual questions. Averaged over all questions, roughly a fifth of the students switched their answers from right to wrong or wrong to right on any given item.

physics.ed-ph↗

arXiv2019

Investigating students' behavior and performance in online conceptual assessment

Historically, the implementation of research-based assessments (RBAs) has been a driver of educational change within physics and helped motivate adoption of interactive engagement pedagogies. Until recently, RBAs were given to students exclusively on paper and in-class; however, this approach has important drawbacks including decentralized data collection and the need to sacrifice class time. Recently, some RBAs have been moved to online platforms to address these limitations. Yet, online RBAs present new concerns such as student participation rates, test security, and students' use of outside resources. Here, we report on a study addressing these concerns in both upper-division and lower-division undergraduate physics courses. We gave RBAs to courses at five institutions; the RBAs were hosted online and featured embedded JavaScript code which collected information on students' behaviors (e.g., copying text, printing). With these data, we examine the prevalence of these behaviors, and their correlation with students' scores, to determine if online and paper-based RBAs are comparable. We find that browser loss of focus is the most common online behavior while copying and printing events were rarer. We found that correlations between these behaviors and student performance varied significantly between introductory and upper-division student populations, particularly with respect to the impact of students copying text in order to utilize internet resources. However, while the majority of students engaged in one or more of the targeted online behaviors, we found that, for our sample, none of these behaviors resulted in a significant change in the population's average performance that would threaten our ability to interpret this performance or compare it to paper-based implementations of the RBA.

physics.ed-ph↗

arXiv2017

Conceptual Assessment Tool for Advanced Undergraduate Electrodynamics

As part of ongoing investigations into student learning in advanced undergraduate courses, we have developed a conceptual assessment tool for upper-division electrodynamics (E&M II): the Colorado UppeR-division ElectrodyNamics Test (CURrENT). This is a free response, post-instruction diagnostic with 6 multi-part questions, an optional 3-question pre-instruction test, and accompanying grading rubrics. The instrument's development was guided by faculty-consensus learning goals and research into common student difficulties. It can be used to gauge the effectiveness of transformed pedagogy, and to gain insights into student thinking in the covered topic areas. We present baseline data representing 500 students across 9 institutions, along with validity, reliability and discrimination measures of the instrument and scoring rubric.

physics.ed-ph↗

arXiv2017

Student Difficulties with Boundary Conditions in Electrodynamics

Identifying and understanding student difficulties with physics content in a wide variety of topical areas is an active research area within the PER community. In many cases, physics topics appear multiple times in different contexts across the undergraduate physics curriculum. As these common topics reappear, student difficulties can perpetuate from one context to the next, or new difficulties can emerge as students encounter new physical contexts. One example of such a topic is boundary conditions. It is broadly considered to be an important topic that advanced physics undergraduates are expected to understand and apply in multiple topical areas including classical mechanics, quantum mechanics, and throughout electricity and magnetism (EM). We report findings from an investigation of student difficulties using boundary conditions focusing on the context of electrodynamics. Our data sources include student responses to traditional exam questions, conceptual survey questions, and think aloud interviews. The analysis was guided by an analytical framework that characterizes how students activate, construct, execute, and reflect on boundary conditions. Common observed student difficulties include: activating boundary conditions in appropriate contexts; constructing a complex expression for the E&M waves; mathematically simplifying and manipulating complex exponential and, checking if the reflection and transmission coefficient are physical. We also present potential pedagogical implications based on our observations.

physics.ed-ph↗

arXiv2016

Assessing Learning Outcomes in Middle-Division Classical Mechanics: The Colorado Classical Mechanics/Math Methods Instrument

Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level Classical Mechanics and Math Methods course (CM 1) at CU Boulder, we have developed a tool to assess student learning of CM 1 concepts in the upper-division. The Colorado Classical Mechanics/Math Methods Instrument (CCMI) builds on faculty consensus learning goals and systematic observations of student difficulties. The result is a 9-question open-ended post-test that probes student learning in the first half of a two-semester classical mechanics / math methods sequence. In this paper, we describe the design and development of this instrument, its validation, and measurements made in classes at CU Boulder and elsewhere.

physics.ed-ph↗

arXiv2015

Student difficulties with Boundary Conditions in electrodynamics

Boundary conditions (BCs) are considered as an important topic that advanced physics under- graduates are expected to understand and apply. We report findings from an investigation of student difficulties using boundary conditions (BCs) in electrodynamics. Our data sources include student responses to traditional exam questions, conceptual survey questions, and think-aloud interviews. The analysis was guided by an analytical framework that characterizes how students activate, con- struct, execute, and reflect on boundary conditions. Common student difficulties include: activating boundary conditions in appropriate contexts; constructing a complex expression for the E&M waves; mathematically simplifying complex exponentials and checking if the reflection and transmission co- efficient are physical. We also present potential pedagogical implications based on our observations.

physics.ed-ph↗

arXiv2015

Validation and analysis of the coupled multiple response Colorado upper-division electrostatics (CUE) diagnostic

Standardized conceptual assessment represents a widely-used tool for educational researchers interested in student learning within the standard undergraduate physics curriculum. For example, these assessments are often used to measure student learning across educational contexts and instructional strategies. However, to support the large-scale implementation often required for cross-institutional testing, it is necessary for these instruments to have question formats that facilitate easy grading. Previously, we created a multiple-response version of an existing, validated, upper-division electrostatics diagnostic with the goal of increasing the instrument's potential for large-scale implementation. Here, we report on the validity and reliability of this new version as an independent instrument. These findings establish the validity of the multiple-response version as measured by multiple test statistics including item difficulty, item discrimination, and internal consistency. Moreover, we demonstrate that the majority of student responses to the new version are internally consistent even when they are incorrect, and provide an example of how the new format can be used to gain insight into student difficulties with specific content in electrostatics.

physics.ed-ph↗

arXiv2015

Upper-division student difficulties with Separation of Variables

Separation of variables can be a powerful technique for solving many of the partial differential equations that arise in physics contexts. Upper-division physics students encounter this technique in multiple topical areas including electrostatics and quantum mechanics. To better understand the difficulties students encounter when utilizing the separation of variables technique, we examined students' responses to midterm exam questions and a standardized conceptual assessment, and conducted think-aloud, problem-solving interviews. Our analysis was guided by an analytical framework that focuses on how students activate, construct, execute, and reflect on the separation of variables technique when solving physics problems. Here we focus on student difficulties with separation of variables as a technique to solve Laplace's equation in both Cartesian and spherical coordinates in the context of junior-level electrostatics. Challenges include: recognizing when separation of variables is the appropriate tool; recalling/justifying the separated form of the potential and the need for the infinite sum; identifying implicit boundary conditions; and spontaneously reflecting on their solutions. Moreover, the type and frequency of errors was often different for SoV problems in Cartesian and spherical geometries. We also briefly discuss implication of these our findings for instruction.

physics.ed-ph↗

arXiv2015

Unpacking Students' Use of Mathematics in Upper-division Physics

In their study of physics beyond the first year of University -- termed upper-division in the US, many of students' primary learning opportunities come from working long, complex back-of-the-book style problems, and from trying to develop an understanding of the underlying physics through solving such problems. Some of the research at the upper-division focuses on how students use mathematics in these problems, and what challenges students encounter along the way. There are a number of different and diverse research studies on students' use of mathematics in the upper-division. These typically utilize one of two broad approaches, with some researchers primarily seeking out and addressing challenges students face, and others working chiefly to unpack students' in-the-moment reasoning. In this paper, we present and discuss both approaches, and then review research efforts that strive to connect these two approaches in order to make sense of students' use of mathematics as well as to uncover particular challenges that students encounter. These recent efforts represent a small step towards synthesizing the two approaches, which we argue is necessary to more meaningfully impact student learning at the upper-division. We close our review and discussion with suggested refinements for future research questions for the physics education research community to consider while it works to understand how students use math in upper-division courses.

physics.ed-ph↗

arXiv2015

Upper-division student difficulties with the Dirac delta function

The Dirac delta function is a standard mathematical tool that appears repeatedly in the undergraduate physics curriculum in multiple topical areas including electrostatics, and quantum mechanics. While Dirac delta functions are often introduced in order to simplify a problem mathematically, students still struggle to manipulate and interpret them. To characterize student difficulties with the delta function at the upper-division level, we examined students' responses to traditional exam questions and a standardized conceptual assessment, and conducted think-aloud interviews. Our analysis was guided by an analytical framework that focuses on how students activate, construct, execute, and reflect on the Dirac delta function in the context of problem solving in physics. Here, we focus on student difficulties using the delta function to express charge distributions in the context of junior-level electrostatics. Common challenges included: invoking the delta function spontaneously, translating a description of a charge distribution into a mathematical expression using delta functions, integrating 3D or non-Cartesian delta function expressions, and recognizing that the delta function can have units. We also briefly discuss implications of these difficulties for instruction.

physics.ed-ph↗