arXiv subjects

Isabel Amigo

Publications and source records attributed to Isabel Amigo.

4 recordsLinked to original sources

arXiv2026

A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment

The integration of artificial intelligence (AI), particularly large language models (LLMs), into educational assessment has opened new opportunities to enhance the efficiency and scalability of grading processes. This study presents the design and validation of an AI-assisted scoring framework for written responses in a large-scale national assessment. The proposed approach focuses on short written texts of approximately 150-200 words and incorporates a human-in-the-loop strategy to preserve assessment quality while reducing manual workload. The study is grounded in a real operational context, using data from two recent editions of a nationwide test, each comprising approximately 5,000 student responses. We analyze the alignment between AI-generated scores and human raters across multiple rubric dimensions, as well as the impact of the proposed decision flow on pass/fail outcomes. Results show moderate to high agreement between the model and human evaluations in most dimensions, supporting the feasibility of AI assistance in this setting. Moreover, the proposed correction workflow identifies cases where human review is most valuable, enabling a more efficient allocation of expert effort. The findings suggest that AI-assisted scoring can be safely integrated into large-scale assessment processes only when combined with carefully designed human oversight. The paper concludes by discussing practical implications for deployment in national assessment systems and outlining future research directions, including longitudinal monitoring of model-human alignment and the analysis of potential cognitive bias introduced by AI-supported review workflows.

cs.CL↗

arXiv2025

On the effectiveness of LLMs for automatic grading of open-ended questions in Spanish

Grading is a time-consuming and laborious task that educators must face. It is an important task since it provides feedback signals to learners, and it has been demonstrated that timely feedback improves the learning process. In recent years, the irruption of LLMs has shed light on the effectiveness of automatic grading. In this paper, we explore the performance of different LLMs and prompting techniques in automatically grading short-text answers to open-ended questions. Unlike most of the literature, our study focuses on a use case where the questions, answers, and prompts are all in Spanish. Experimental results comparing automatic scores to those of human-expert evaluators show good outcomes in terms of accuracy, precision and consistency for advanced LLMs, both open and proprietary. Results are notably sensitive to prompt styles, suggesting biases toward certain words or content in the prompt. However, the best combinations of models and prompt strategies, consistently surpasses an accuracy of 95% in a three-level grading task, which even rises up to more than 98% when the it is simplified to a binary right or wrong rating problem, which demonstrates the potential that LLMs have to implement this type of automation in education applications.

cs.CL↗

arXiv2023

Stability of local tip pool sizes

In directed acyclic graph (DAG)-based distributed ledgers, unreferenced blocks (tips) form the backlog of a distributed queueing system. Each new block creates one tip and attempts to remove up to $k$ existing tips by referencing them. With heterogeneous propagation delays, these service decisions are made from delayed local information, so nodes may disagree on the backlog and some reference attempts are wasted. We study a continuous-time Poisson model with bounded heterogeneous delays and uniform tip selection. We prove that the embedded tip-configuration chain is irreducible, aperiodic, and positive Harris recurrent, and hence admits a unique stationary regime. The observer and local tip-pool sizes have stationary exponential moments, converge to their stationary limits, and satisfy almost-sure ergodic averages. We also derive a Little-type identity relating the stationary mean observer tip count to the mean time until a typical block is first referenced. Simulations are included as qualitative illustrations of the effects of delay variability and issuance heterogeneity.

math.PR↗

arXiv2022

Online Multi-Agent Decentralized Byzantine-robust Gradient Estimation

In this paper, we propose an iterative scheme for distributed Byzantineresilient estimation of a gradient associated with a black-box model. Our algorithm is based on simultaneous perturbation, secure state estimation and two-timescale stochastic approximations. We also show the performance of our algorithm through numerical experiments.

cs.AI↗