arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2603.20220

Development and Validation of a Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale for Higher Education

Yukyeong Song·Hyunjoo Moon·Hyewon Yang·Chris Kilgore
View original source ↗Read source document ↗Export metadata ↓

Abstract

The integration of artificial intelligence (AI) in higher education underscores the growing importance of faculty AI literacy and competency across teaching, research, and service. Existing AI literacy instruments, however, primarily target the general public, students, or K-12 teachers, and therefore lack the role-embedded indicators and psychometric validation needed for scalable assessment among university faculty. Grounded in the Critical Tech-resilient Literacies (CTRL) framework, this study develops and validates the Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale as a concise and practically deployable tool for higher education contexts. Using a theory-driven development process, we generated an initial pool of 43 items mapped to three literacies (functional, evaluative, and ethical literacy) and situated them across four faculty work domains (general, teaching, research, service/administration), creating a 3 x 4 framework. Content validation was conducted through structured interviews with four subject-matter experts, supplemented by a GPT-based reviewer to triangulate ratings of clarity, relevance, and necessity, yielding refined 39 items for pilot testing. Pilot testing involved 269 valid responses, which were analyzed using confirmatory factor analysis (CFA). CFA evaluated the theoretically specified structure, followed by item reduction to minimize respondent burden while preserving content coverage. The final 23-item FALCON-AI demonstrated good model fit for the AI Literacy x Faculty Work measurement and strong reliability. This study presents a validated FALCON-AI scale with good reliability and validity, offering a refined practical instrument for assessing faculty AI in higher education.

Explore related subjects

cs.CY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yukyeong Song, Hyunjoo Moon, Hyewon Yang, Chris Kilgore. 2026-03-03. Development and Validation of a Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale for Higher Education. https://arxiv.org/abs/2603.20220

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Towards interactive evaluations for interaction harms in human-AI systems

Current AI evaluation methods, which rely on static, model-only tests, fail to account for harms that emerge through sustained human-AI interaction. As AI systems proliferate and are increasingly integrated into real-world applications, this disconnect between evaluation approaches and actual usage becomes more significant. In this paper, we propose a shift towards evaluation based on \textit{interactional ethics}, which focuses on \textit{interaction harms} - issues like inappropriate parasocial relationships, social manipulation, and cognitive overreliance that develop over time through repeated interaction, rather than through isolated outputs. First, we discuss the limitations of current evaluation methods, which (1) are static, (2) assume a universal user experience, and (3) have limited construct validity. Drawing on research from human-computer interaction, natural language processing, and the social sciences, we present practical principles for designing interactive evaluations. These include ecologically valid interaction scenarios, human impact metrics, and diverse human participation approaches. Finally, we explore implementation challenges and open research questions for researchers, practitioners, and regulators aiming to integrate interactive evaluations into AI governance frameworks. This work lays the groundwork for developing more effective evaluation methods that better capture the complex dynamics between humans and AI systems.

cs.CY↗
arXiv2026

Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

LLMs have shown strong performance in data-rich domains such as programming, but their reliability in engineering tasks remains limited. Circuit analysis is particularly challenging because it requires both multimodal understanding and precise mathematical reasoning. This paper presents an enhanced end-to-end circuit problem-solving framework using Gemini 2.5 Pro as the backbone model for scalable engineering-education applications. We systematically evaluate Gemini 2.5 Pro on undergraduate circuit-analysis problems and identify two major failure modes: circuit-recognition hallucinations, especially source-polarity errors, and reasoning-process hallucinations, such as incorrect current-direction assumptions. To reduce recognition errors, we integrate a fine-tuned YOLO detector with OpenCV-based processing to isolate voltage and current sources for polarity re-identification. To mitigate reasoning errors, we introduce an ngspice-driven verification loop that supports iterative refinement with optional human feedback. On 83 problems, the proposed pipeline achieves 97.59% accuracy, compared with 79.52% for baseline Gemini. Across four hand-drawn diagram variations, accuracy improves from 60.61%--71.21% to 89.39%--92.42%, with statistically significant gains (p<0.005). On 43 problems from a different textbook, accuracy increases from 58.14% to 83.72%, further supporting cross-textbook generalizability. Error analysis shows that circuit recognition remains the dominant source of residual failures, particularly under varying diagram representations. Overall, the framework substantially improves the robustness, scalability, and generalizability of LLM-based circuit problem solving for engineering education and practical circuit analysis.

cs.CY↗
arXiv2026

GreenPassport: Request-Level Carbon Accounting for Cross-Border AI Inference

AI inference often crosses regional boundaries as prompts travel to remote data centers and generated tokens return to users. Regional averages cannot represent the resulting differences in serving hardware, electricity, and network delivery. Request-level accounting needs a common boundary for the service, serving site, route, local comparator, uncertainty, and data provenance. GreenPassport Carbon Accounting (GPCA) associates these inputs with each request. It estimates serving and route carbon, then selects a reporting level from the available documentation. Our public-data implementation covers data-center instances, accelerators, model families, electricity mixes, routes, and cloud-region carbon intensity. Against six accounting baselines and four energy-prediction baselines, GPCA reduced median absolute percentage error by 56.3% and median absolute error by 15.5% relative to EcoLogits under the aligned accelerator-energy boundary. It produced zero rule overstatement in the deterministic conformance tests. In the buyer case, the clean-electricity CN-West scenario produced 0.0148 gCO2e per request, 88% below the local service at 0.1220 gCO2e per request.

cs.CY↗