arXiv subjects

Stephen MacNeil

Publications and source records attributed to Stephen MacNeil.

At least 19 recordsLinked to original sources

arXiv2026

Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks

Learning to communicate with code-generating AI models is an emerging skill for novice programmers. One recent pedagogical approach, Prompt Problems, has students solve computational tasks by writing natural-language prompts for code-generating AI models. However, little is known about the specific prompt-level mistakes novice programmers make, the kinds of computational details they fail to communicate, and what strategies they use to recover when generated code is incorrect. In a CS1 course, we studied attempts by more than 900 students to solve dialogue-based Prompt Problems. We analyzed student reflections, unsuccessful prompts, and reported debugging strategies. Compared to traditional coding tasks, students generally found prompting easier, more enjoyable, and better targeted at developing problem-solving skills. The most common mistakes are related to the omission of key details, suggesting both a failure to acknowledge their importance and over-reliance on AI to infer them. When prompts failed, students focused more on clarifying their intent and reflecting on the provided problem details than on tracing generated code or examining test cases.

cs.CY↗

arXiv2026

To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks

AI code completion tools, such as Github Copilot, provide students with code suggestions to help them write programs. However, recent qualitative studies suggest that students fail to critically evaluate these suggestions. We present Clover, a code completion tool that logs students' interactions with code suggestions and additionally offers attention checks to probe reflective engagement during programming tasks. We also develop a taxonomy of behavioral interaction metrics for AI-assisted programming, informed by literature. We analyzed relationships between interaction patterns, engagement with attention checks, and task performance. We observed that higher rates of tab accept were associated with lower attention check performance, while increased dwell time was associated with higher attention check performance. We conclude by discussing how programming process data and attention checks might support reflective engagement in AI-assisted programming.

cs.HC↗

arXiv2026

Exploring the Value of Diverse LLM Explanations in Introductory Programming

Large Language Models (LLMs) have shown the potential to generate code explanations that surpass those of peers in quality, offering promising opportunities for computer science education. While these explanations may not yet match the depth and clarity of instructor-provided explanations, research in computational creativity highlights that the quantity and diversity of ideas can often outweigh a singular focus on quality. Inspired by this, we explore whether combining multiple diverse explanations, each emphasizing distinct aspects (e.g., function, concept, goal), can enhance students' understanding of programming exercises compared to generic explanations that do not emphasize distinct conceptual aspects. In our study 971 first-year computing students were randomly assigned either diverse or generic LLM-generated explanations for two programming exercises. Students completed multiple-choice and open-ended questions for each exercise, followed by Likert-scale questions and open-ended reflections. Our findings outline patterns in student performance and perceived cognitive load across the two explanation conditions. These findings highlight how variation in explanation emphasis may relate to learner engagement and understanding. Across participants, open-ended response accuracy was consistently about 7.7% higher when students received diverse explanations, with no difference in perceived cognitive load.

cs.HC↗

arXiv2026

"Stuck in a Spiral": Shame and Guilt as Social Regulators of AI Use in Computing Education

While prior work has examined patterns of adoption and social norms around AI use, less is known about how emotional factors, such as shame and guilt, shape students use of AI tools. We present an interview study with 19 computing students through a functionalist perspective of shame and guilt, which interprets emotions as social signals that regulate behavior. Our findings show that these emotions regulate when and how students make their use visible, as they engage in hiding behaviors and selective disclosure. Students described shaming themselves, their peers, and even faculty for using AI. Shame and guilt often coexist with continued AI use, creating cycles of reduced agency and moral tension rather than promoting behavior change. Students described feeling tensions between their AI use and their identities as competent, hardworking, or ethical computing students. Students also used language and metaphors of addiction to describe their experiences. These results highlight the need to consider the socio-emotional aspects of AI use, which may be influenced by how AI policies are implemented and enforced. We discuss classroom practices that can foster healthy, open discussion and support responsible AI use.

cs.CY↗

arXiv2026

"I can't read your mind": A Study of Neurodivergent Computing Students' Experiences with Collaborative Active Learning

Computing courses often feature active learning techniques that promote collaboration and social interaction between students. However, neurodivergent students' preferences and experiences with these techniques are not well understood. We conducted a survey of neurodivergent computing students (n=24), specifically autistic students or students with ADHD, and neurotypical computing students (n=20) to understand how the structure of collaborative active learning affects their comfort in computing courses. We also interviewed four computing students on the autism spectrum or with ADHD to gain more contextualized insights into their experiences and accessibility recommendations. Our survey surfaces how team dynamics and assignment structure can impact neurodivergent students' comfort in computing courses. Neurodivergent students expressed discomfort with assignments that lack structure or have ambiguous expectations. Neurodivergent students prefer smaller teams that work together frequently with explicitly defined roles. Our interviews identified ways that neurodivergent students cope with discomfort in collaborative active learning, including self-selecting roles and self-disclosure. While preliminary, our results highlight how instructors can design collaborative active learning to be more equitable and accessible for neurodivergent students.

cs.HC↗

arXiv2026

Changing the Optics: Comparing Traditional and Retrieval-Augmented GenAI E-Tutorials in Interdisciplinary Learning

Understanding information-seeking behaviors in e-learning is critical, as learners must often make sense of complex and fragmented information, a challenge compounded in interdisciplinary fields with diverse prior knowledge. Compared to traditional e-tutorials, GenAI e-tutorials offer new ways to navigate information spaces, yet how they shape learners information-seeking behaviors remains unclear. To address this gap, we characterized behavioral differences between traditional and GenAI-mediated e-tutorial learning using the three search modes of orienteering. We conducted a between-subject study in which learners engaged with either a traditional e-tutorial or a GenAI e-tutorial accessing the same underlying information content. We found that the traditional users maintained greater awareness and focus of the information space, whereas GenAI users exhibited more proactive and exploratory behaviors with lower cognitive load due to the querying-driven interaction. These findings offer guidance for designing tutorials in e-learning.

cs.HC↗

arXiv2025

Navigating the Ethical and Societal Impacts of Generative AI in Higher Computing Education

Generative AI (GenAI) presents societal and ethical challenges related to equity, academic integrity, bias, and data provenance. In this paper, we outline the goals, methodology and deliverables of their collaborative research, considering the ethical and societal impacts of GenAI in higher computing education. A systematic literature review that addresses a wide set of issues and topics covering the rapidly emerging technology of GenAI from the perspective of its ethical and societal impacts is presented. This paper then presents an evaluation of a broad international review of a set of university adoption, guidelines, and policies related to the use of GenAI and the implications for computing education. The Ethical and Societal Impacts-Framework (ESI-Framework), derived from the literature and policy review and evaluation, outlines the ethical and societal impacts of GenAI in computing education. This work synthesizes existing research and considers the implications for computing higher education. Educators, computing professionals and policy makers facing dilemmas related to the integration of GenAI in their respective contexts may use this framework to guide decision-making in the age of GenAI.

cs.CY↗

arXiv2025

Beyond the Benefits: A Systematic Review of the Harms and Consequences of Generative AI in Computing Education

Generative artificial intelligence (GenAI) has already had a big impact on computing education with prior research identifying many benefits. However, recent studies have also identified potential risks and harms. To continue maximizing AI benefits while addressing the harms and unintended consequences, we conducted a systematic literature review of research focusing on the risks, harms, and unintended consequences of GenAI in computing education. Our search of ACM DL, IEEE Xplore, and Scopus (2022-2025) resulted in 1,677 papers, which were then filtered to 224 based on our inclusion and exclusion criteria. Guided by best practices for systematic reviews, four reviewers independently extracted publication year, learner population, research method, contribution type, GenAI technology, and educational task information from each paper. We then coded each paper for concrete harm categories such as academic integrity, cognitive effects, and trust issues. Our analysis shows patterns in how and where harms appear, highlights methodological gaps and opportunities for more rigorous evidence, and identifies under-explored harms and student populations. By synthesizing these insights, we intend to equip educators, computing students, researchers, and developers with a clear picture of the harms associated with GenAI in computing education.

cs.HC↗

arXiv2025

Helping or Homogenizing? GenAI as a Design Partner to Pre-Service SLPs for Just-in-Time Programming of AAC

Augmentative and alternative communication (AAC) devices are used by many people around the world who experience difficulties in communicating verbally. One AAC device which is especially useful for minimally verbal autistic children in developing language and communication skills are visual scene displays (VSD). VSDs use images with interactive hotspots embedded in them to directly connect language to real-world contexts which are meaningful to the AAC user. While VSDs can effectively support emergent communicators, their widespread adoption is impacted by how difficult these devices are to configure. We developed a prototype that uses generative AI to automatically suggest initial hotspots on an image to help non-experts efficiently create VSDs. We conducted a within-subjects user study to understand how effective our prototype is in supporting non-expert users, specifically pre-service speech-language pathologists (SLP) who are not familiar with VSDs as an AAC intervention. Pre-service SLPs are actively studying to become clinically certified SLPs and have domain-specific knowledge about language and communication skill development. We evaluated the effectiveness of our prototype based on creation time, quality, and user confidence. We also analyzed the relevance and developmental appropriateness of the automatically generated hotspots and how often users interacted with the generated hotspots. Our results were mixed with SLPs becoming more efficient and confident. However, there were multiple negative impacts as well, including over-reliance and homogenization of communication options. The implications of these findings reach beyond the domain of AAC, especially as generative AI becomes more prevalent across domains, including assistive technology. Future work is needed to further identify and address these risks associated with integrating generative AI into assistive technology.

cs.HC↗

arXiv2025

Conversations over Clicks: Impact of Chatbots on Information Search in Interdisciplinary Learning

This full research paper investigates the impact of generative AI (GenAI) on the learner experience, with a focus on how learners engage with and utilize the information it provides. In e-learning environments, learners often need to navigate a complex information space on their own. This challenge is further compounded in interdisciplinary fields like bioinformatics, due to the varied prior knowledge and backgrounds. In this paper, we studied how GenAI influences information search in bioinformatics research: (1) How do interactions with a GenAI chatbot influence learner orienteering behaviors?; and (2) How do learners identify information scent in GenAI chatbot responses? We adopted an autoethnographic approach to investigate these questions. GenAI was found to support orienteering once a learning plan was established, but it was counterproductive prior to that. Moreover, traditionally value-rich information sources such as bullet points and related terms proved less effective when applied to GenAI responses. Information scents were primarily recognized through the presence or absence of prior knowledge of the domain. These findings suggest that GenAI should be adopted into e-learning environments with caution, particularly in interdisciplinary learning contexts.

cs.HC↗

arXiv2025

Feedstack: Layering Structured Representations over Unstructured Feedback to Scaffold Human AI Conversation

Many conversational user interfaces facilitate linear conversations with turn-based dialogue, similar to face-to-face conversations between people. However, digital conversations can afford more than simple back-and-forth; they can be layered with interaction techniques and structured representations that scaffold exploration, reflection, and shared understanding between users and AI systems. We introduce Feedstack, a speculative interface that augments feedback conversations with layered affordances for organizing, navigating, and externalizing feedback. These layered structures serve as a shared representation of the conversation that can surface user intent and reveal underlying design principles. This work represents an early exploration of this vision using a research-through-design approach. We describe system features and design rationale, and present insights from two formative (n=8, n=8) studies to examine how novice designers engage with these layered supports. Rather than presenting a conclusive evaluation, we reflect on Feedstack as a design probe that opens up new directions for conversational feedback systems.

cs.HC↗

arXiv2025

Lightweight Social Computing Tools for Undergraduate Research Community Building

Many barriers exist when new members join a research community, including impostor syndrome. These barriers can be especially challenging for undergraduate students who are new to research. In our work, we explore how the use of social computing tools in the form of spontaneous online social networks (SOSNs) can be used in small research communities to improve sense of belonging, peripheral awareness, and feelings of togetherness within an existing CS research community. Inspired by SOSNs such as BeReal, we integrated a Wizard-of-Oz photo sharing bot into a computing research lab to foster community building among members. Through a small sample of lab members (N = 17) over the course of 2 weeks, we observed an increase in participants' sense of togetherness based on pre- and post-study surveys. Our surveys and semi-structured interviews revealed that this approach has the potential to increase awareness of peers' personal lives, increase feelings of community, and reduce feelings of disconnectedness.

cs.SI↗

arXiv2025

Neurodiversity in Computing Education Research: A Systematic Literature Review

Ensuring equitable access to computing education for all students-including those with autism, dyslexia, or ADHD-is essential to developing a diverse and inclusive workforce. To understand the state of disability research in computing education, we conducted a systematic literature review of research on neurodiversity in computing education. Our search resulted in 1,943 total papers, which we filtered to 14 papers based on our inclusion criteria. Our mixed-methods approach analyzed research methods, participants, contribution types, and findings. The three main contribution types included empirical contributions based on user studies (57.1%), opinion contributions and position papers (50%), and survey contributions (21.4%). Interviews were the most common methodology (75% of empirical contributions). There were often inconsistencies in how research methods were described (e.g., number of participants and interview and survey materials). Our work shows that research on neurodivergence in computing education is still very preliminary. Most papers provided curricular recommendations that lacked empirical evidence to support those recommendations. Three areas of future work include investigating the impacts of active learning, increasing awareness and knowledge about neurodiverse students' experiences, and engaging neurodivergent students in the design of pedagogical materials and computing education research.

cs.HC↗

arXiv2025

Probing the Unknown: Exploring Student Interactions with Probeable Problems at Scale in Introductory Programming

Introductory programming courses often rely on small code-writing exercises that have clearly specified problem statements. This limits opportunities for students to practice how to clarify ambiguous requirements -- a critical skill in real-world programming. In addition, the emerging capabilities of large language models (LLMs) to produce code from well-defined specifications may harm student engagement with traditional programming exercises. This study explores the use of ``Probeable Problems'', automatically gradable tasks that have deliberately vague or incomplete specifications. Such problems require students to submit test inputs, or `probes', to clarify requirements before implementation. Through analysis of over 40,000 probes in an introductory course, we identify patterns linking probing behaviors to task success. Systematic strategies, such as thoroughly exploring expected behavior before coding, resulted in fewer incorrect code submissions and correlated with course success. Feedback from nearly 1,000 participants highlighted the challenges and real-world relevance of these tasks, as well as benefits to critical thinking and metacognitive skills. Probeable Problems are easy to set up and deploy at scale, and help students recognize and resolve uncertainties in programming problems.

cs.HC↗

arXiv2025

Exploring Student Behaviors and Motivations using AI TAs with Optional Guardrails

AI-powered chatbots and digital teaching assistants (AI TAs) are gaining popularity in programming education, offering students timely and personalized feedback. Despite their potential benefits, concerns about student over-reliance and academic misconduct have prompted the introduction of "guardrails" into AI TAs - features that provide scaffolded support rather than direct solutions. However, overly restrictive guardrails may lead students to bypass these tools and use unconstrained AI models, where interactions are not observable, thus limiting our understanding of students' help-seeking behaviors. To investigate this, we designed and deployed a novel AI TA tool with optional guardrails in one lab of a large introductory programming course. As students completed three code writing and debugging tasks, they had the option to receive guardrailed help or use a "See Solution" feature which disabled the guardrails and generated a verbatim response from the underlying model. We investigate students' motivations and use of this feature and examine the association between usage and their course performance. We found that 50% of the 885 students used the "See Solution" feature for at least one problem and 14% used it for all three problems. Additionally, low-performing students were more likely to use this feature and use it close to the deadline as they started assignments later. The predominant factors that motivated students to disable the guardrails were assistance in solving problems, time pressure, lack of self-regulation, and curiosity. Our work provides insights into students' solution-seeking motivations and behaviors, which has implications for the design of AI TAs that balance pedagogical goals with student preferences.

cs.HC↗

arXiv2025

"All Roads Lead to ChatGPT": How Generative AI is Eroding Social Interactions and Student Learning Communities

The widespread adoption of generative AI is already impacting learning and help-seeking. While the benefits of generative AI are well-understood, recent studies have also raised concerns about increased potential for cheating and negative impacts on students' metacognition and critical thinking. However, the potential impacts on social interactions, peer learning, and classroom dynamics are not yet well understood. To investigate these aspects, we conducted 17 semi-structured interviews with undergraduate computing students across seven R1 universities in North America. Our findings suggest that help-seeking requests are now often mediated by generative AI. For example, students often redirected questions from their peers to generative AI instead of providing assistance themselves, undermining peer interaction. Students also reported feeling increasingly isolated and demotivated as the social support systems they rely on begin to break down. These findings are concerning given the important role that social interactions play in students' learning and sense of belonging.

cs.CY↗

arXiv2025

The Role of Generative AI in Software Student CollaborAItion

Collaboration is a crucial part of computing education. The increase in AI capabilities over the last couple of years is bound to profoundly affect all aspects of systems and software engineering, including collaboration. In this position paper, we consider a scenario where AI agents would be able to take on any role in collaborative processes in computing education. We outline these roles, the activities and group dynamics that software development currently include, and discuss if and in what way AI could facilitate these roles and activities. The goal of our work is to envision and critically examine potential futures. We present scenarios suggesting how AI can be integrated into existing collaborations. These are contrasted by design fictions that help demonstrate the new possibilities and challenges for computing education in the AI era.

cs.SE↗

arXiv2024

The Evolving Usage of GenAI by Computing Students

Help-seeking is a critical aspect of learning and problem-solving for computing students. Recent research has shown that many students are aware of generative AI (GenAI) tools; however, there are gaps in the extent and effectiveness of how students use them. With over two years of widespread GenAI usage, it is crucial to understand whether students' help-seeking behaviors with these tools have evolved and how. This paper presents findings from a repeated cross-sectional survey conducted among computing students across North American universities (n=95). Our results indicate shifts in GenAI usage patterns. In 2023, 34.1% of students (n=47) reported never using ChatGPT for help, ranking it fourth after online searches, peer support, and class forums. By 2024, this figure dropped sharply to 6.3% (n=48), with ChatGPT nearly matching online search as the most commonly used help resource. Despite this growing prevalence, there has been a decline in students' hourly and daily usage of GenAI tools, which may be attributed to a common tendency to underestimate usage frequency. These findings offer new insights into the evolving role of GenAI in computing education, highlighting its increasing acceptance and solidifying its position as a key help resource.

cs.CY↗