arXiv subjects

Anas Madkoor

Publications and source records attributed to Anas Madkoor.

3 recordsLinked to original sources

arXiv2026

QuranicMMLU: A Cognitively-Aware Benchmark for Evaluating Generative AI Solutions on Quranic Linguistic Knowledge

We introduce QuranicMMLU, a benchmark for evaluating generative AI on Quranic Arabic across multiple dimensions of linguistic complexity. Existing Quranic benchmarks center on general question answering and semantic retrieval, without probing specific linguistic competencies or stratifying by cognitive demand and verse difficulty. We construct a five-pillar Quranic taxonomy spanning Phonology, Morphology, Syntax, Semantics, and Pragmatics, with 31 leaves covering phenomena from tajwīd and root-and-pattern morphology to occasions of revelation and inter-surah coherence. For each leaf we generate questions stratified by Bloom's cognitive level and verse perplexity, then have LLM as a judge to independently answer and score every item and route the annotations to manual review. The resulting dataset comprises 980 human-reviewed questions, each issued in both open-ended and multiple-choice form. We benchmark 12 systems on these items and find that the Islamic-specialized model leads, yet every system scores higher on multiple-choice accuracy (average 84%) than open-ended answer quality (average 60%): the two rankings agree closely (Kendall's τ=0.73), but multiple-choice scoring hides failures that surface only once answer choices are removed. QuranicMMLU thus offers a rigorous, linguistically grounded framework for evaluating Arabic NLP in the Quranic domain.

cs.CL↗

arXiv2026

Virtual Reality-Simulated Interaction Between Micro-Mobility Vehicles and Pedestrians: A Biomechanical Analysis of Human Gait and Movement Responses

Pedestrian walking is a fundamental activity of daily living and a key component of first and last-mile urban mobility. The rapid adoption of e-scooters has increased pedestrian-vehicle interactions on shared sidewalks and crossings, raising collision risks. However, most previous studies have relied on trajectory-based observations, providing limited insight into biomechanical gait responses. This study investigated pedestrian gait adaptations during simulated e-scooter interactions using immersive virtual reality (VR) and markerless pose estimation. Twelve healthy male university students (21-23 years) completed four VR walking scenarios: normal walking, e-scooter encounters at 10-25 km/h, crossing encounters, and near-crash encounters. Sagittal-plane videos were analyzed using the OpenPose 25-point model. Step length, gait cycle time, walking velocity, stance and swing phases, and lower-limb joint trajectories were extracted using Kinovea and custom JSON-based analysis tools. Statistical analyses included ANOVA, MANOVA, and non-parametric tests Crossing and near-crash scenarios significantly reduced step length (p<0.001), from 226.5 cm during normal walking to 204.7 cm during near-crash simulations. Although gait velocity and timing were not significantly affected, participants consistently exhibited shorter stance phases, longer swing phases, and restricted knee motion during stressful encounters, indicating reflexive gait adaptations to perceived collision risk. These findings demonstrate that immersive VR combined with markerless pose estimation effectively quantifies pedestrian biomechanical responses to micro-mobility interactions. Gait adaptations identified in this study may serve as sensitive indicators of collision risk and support the development of proactive pedestrian safety measures and intelligent micro-mobility control systems.

eess.SP↗

arXiv2026

Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models

Despite the rapid progress of Vision-Language Models (VLMs), the field lacks benchmarks that rigorously diagnose their true reasoning abilities and chart meaningful progress toward human-like multimodal intelligence. Most existing evaluations focus on piecemeal or disconnected tasks, obscuring critical cognitive weaknesses and providing little insight for targeted improvement. To address this gap, we introduce BloomBench, part of the Almieyar benchmarking series, the first cognitively human-grounded, bilingual (English-Arabic) multimodal benchmark for VLMs. Grounded in Bloom's Taxonomy, BloomBench systematically evaluates six levels of cognition (Remember, Understand, Apply, Analyze, Evaluate, Create) through carefully designed image-question-answer tasks. Built with a semi-automated pipeline and validated through a stratified hybrid quality assurance protocol, it ensures scalability, cultural inclusivity, and linguistic fidelity. Leveraging this framework, we conduct a comprehensive study of state-of-the-art VLMs to diagnose their cognitive profiles. Our analysis reveals a sharp cognitive asymmetry: while state-of-the-art models achieve strong performance ceilings in semantic understanding, they struggle substantially with factual recall and creative synthesis. This demonstrates that current general multimodal proficiency masks deeper limitations in specific cognitive layers. Furthermore, our study highlights a critical performance gap between Arabic and English, exposing limitations in current cross-lingual multimodal reasoning. These findings establish a foundation for developing more cognitively aligned and inclusive VLMs. The benchmark framework and dataset is available at: https://github.com/qcri/Almieyar-Oryx-BloomBench.

cs.CV↗