Supporting Students in Navigating LLM-Generated Insecure Code

Abstract

The advent of Artificial Intelligence (AI), particularly large language models (LLMs), has revolutionized software development by enabling developers to specify tasks in natural language and receive corresponding code, boosting productivity. However, this shift also introduces security risks, as LLMs may generate insecure code that can be exploited by adversaries. Conventional educational approaches emphasize efficiency while overlooking these risks, leaving students unprepared to identify and mitigate security issues in AI-assisted workflows. To surface this gap, we present \texttt{Bifröst}, a classroom measurement and feedback framework that pairs an adversarially configured code-generation model with a VS Code extension and automated vulnerability reports that instructors can use to guide follow-up discussions. Through classroom deployments with undergraduate students ($n=61$), we observe that students frequently accepted insecure LLM-generated code despite prior security coursework and stated skepticism. A post-feedback survey ($n=21$) provides preliminary evidence that students' stated trust shifted toward greater skepticism after receiving Bifröst feedback, and that some students articulated more security-specific concerns about AI-generated code.