arXiv ScienceSearch

arXiv subjects

Sagar Dasgupta

Publications and source records attributed to Sagar Dasgupta.

At least 19 recordsLinked to original sources

LLMSec-AV: A Vulnerability Taxonomy and LLM-Driven Software Weakness Discovery Framework for Autonomous Vehicles

Automated vehicles rely on millions of lines of safety-critical software, yet general-purpose analyzers do not understand which code can affect vehicle motion. This study asks whether large language models (LLMs) with explicit automated-vehicle (AV) security knowledge improve weakness detection beyond rule-based tools. We developed an AV vulnerability taxonomy with 18 weakness classes from vulnerability records, security advisories, and AV-security literature, and integrated it into LLM-based Security Analysis for Automated Vehicles (LLMSec-AV). Evaluated on Autoware, the framework decomposed 770 translation units into 4,673 functions and analyzed 161 functions under four prompting conditions involving taxonomy context, retrieval from 374 prior disclosures, and multi-step analysis. Findings were compared with 46 weakness locations mined from upstream fixes and a flag-volume-matched permutation baseline. CodeQL, Semgrep, cppcheck, and the Clang Static Analyzer evaluated the same code, with AV-specific rules added to CodeQL and Semgrep. Generated fuzzing harnesses were tested using AFL++ and sanitizers. LLM conditions recovered up to 76% of the 46 known weakness locations, outperforming conventional analyzers. CodeQL, Semgrep, and the Clang Static Analyzer matched none, while cppcheck matched one despite 1,301 alerts. Unaided prompting achieved similar detection performance, showing that the taxonomy did not drive recall. However, taxonomy context increased the share of findings assigned to a weakness class from near zero to over 80%, improving interpretability and triage. Six of the 18 classes could not be directly represented as static-analysis rules. LLMSec-AV introduces an AV-specific, machine-readable vulnerability taxonomy for weakness discovery and shows that LLMs can complement conventional analyzers by identifying and organizing safety-relevant findings in real AV software.

cs.CR

Structured Driving-State Narratives for Small Language Model-Based GNSS Spoofing Detection

Autonomous vehicles (AVs) depend on reliable Global Navigation Satellite System (GNSS) positioning. However, spoofed GNSS signals can induce plausible but incorrect vehicle states. This study develops a small language model (SLM)-based framework for detecting and classifying GNSS spoofing attacks by comparing vehicle behaviors independently derived from GNSS and other sensing sources. The framework converts independent driving states from GNSS and other sensing sources into structured semantic narratives that are provided to an SLM for spoofing detection and attack classification. The performance of the SLM-based framework is compared with large language models (LLMs) fine-tuned on identical training data and evaluated on the same test set. The evaluation considers five classes: no attack, overshoot attack, stopped attack, turn-by-turn attack, and wrong-turn attack. The framework is also evaluated with geographically unseen field data collected in Clemson, South Carolina, United States. Experimental results indicate that the evaluated SLMs achieve performance similar to the LLMs, achieving an average accuracy of 96.99%, precision of 99.05%, recall of 95.59%, and F1-score of 97.18%. In terms of computational efficiency and resource utilization, the SLMs demonstrate advantages over the LLMs by requiring lower inference latency and less GPU memory during both fine-tuning and inference. Evaluation using field data collected in a geographically distinct location further demonstrated its efficacy. The presented framework can detect and classify GNSS spoofing attacks in real-time while requiring relatively low computational and memory resources, and is therefore suitable for deployment on resource-constrained vehicular computing platforms.

cs.CR

LLM-Assisted Dynamic Threat Analysis for Attacker-Reachable Software Weaknesses in Autonomous Vehicles

Autonomous vehicles depend on large safety-critical software stacks, where weaknesses reachable from adversarial inputs may affect steering, braking, or other control decisions. Static analysis can identify candidate sites, but dynamically confirming exploitability requires executable test artifacts that are difficult to construct manually. We investigate whether large language models (LLMs) can automate this process for Autoware, an open-source autonomous-driving stack. We perform compiler-precise static analysis across 185 packages, identifying 1,375 decision rules, 2,274 validation checks, and 482 input-to-safety-output flows, from which we derive a weakness taxonomy and sample 740 reachable sites. Two local open-weight LLMs, a no-static-context ablation, and a naive-template baseline generate 3,700 artifact sets, which are compiled against the real build under sanitizers, repaired through compiler-in-the-loop feedback, and fuzzed when executable. The main result is a build-integration failure taxonomy showing that 80% of first-shot compilation failures arise from dependency wiring rather than program logic. The reasoning model compiled 64% of harnesses on the first attempt, compared with 6% for the code-specialized model. Repair achieved full object-compileability for the reasoning model only through extensive stubbing; fewer than half of its harnesses reached the fuzzer, and all 37 observed crashes originated in stubbed code rather than Autoware. No candidate weakness was dynamically confirmed within budget. These results show that build integration, not candidate generation or fuzzing, is the primary barrier to reliable LLM-assisted dynamic analysis of full autonomous-vehicle software stacks.

cs.SE

Treating Statewide CORS Networks as Spatially Distributed Sensors for GNSS Integrity Monitoring under Unintentional and Deliberate Threats

State departments of transportation (DOTs) in the United States increasingly rely on statewide continuously operating reference station (CORS) networks to support high-precision Global Navigation Satellite System (GNSS)-based positioning and timing for intelligent transportation systems. These networks also provide continuous observations that can support regional GNSS integrity monitoring. This study develops and demonstrates a framework that treats a statewide CORS network as a spatially distributed sensor system for identifying unintentional (environmental) and intentional (cyber) interference when GNSS measurements deviate from expected spatial patterns. We develop a graph-based Network Consistency Framework (NCF) that evaluates each station against its spatial neighborhood using four metrics: neighborhood residual, spatial gradient, residual, and graph smoothness. These metrics are combined into a Network Consistency Index (NCI). The framework is demonstrated using two consecutive days of four-constellation observations from 50 stations in the Alabama DOT-maintained CORS network, using changes in vertical total electron content ({\Delta}VTEC) and the Rate of TEC Index (ROTI) as spatially coherent observables. The framework quantified network-wide spatial consistency and identified localized anomalies. Detected anomalies indicate stations whose observations deviated from the surrounding regional network, signaling potential integrity issues. Determining whether anomalies result from receiver faults, localized interference, spoofing, or other causes requires further investigation. This study introduces statewide CORS networks as regional GNSS integrity observatories and presents the NCF and NCI for graph-based spatial integrity monitoring. Transportation agencies can implement the framework using existing CORS observations to monitor network integrity and identify localized anomalies.

cs.CR

Deployment Feasibility Analysis of Post-Quantum Digital Signatures in Safety-Critical C-V2X Communication for Urban Mobility Scenario

The transition from the classical ECDSA to PQC creates substantially larger authentication payloads for safety-critical C-V2X sidelink communication. This study determines which NIST post-quantum signature algorithms are compatible with the current SAE J3161 deployment profile and quantifies their communication-level effects. A transport-block feasibility analysis was performed using IEEE 1609.2 secured-message structures, SAE J3161 radio parameters, and the signature and public-key sizes of ECDSA P-256, Falcon-512, Dilithium-2, and SPHINCS+. Falcon-512, the only post-quantum candidate that fit the applicable transport-block constraints, was compared with ECDSA P-256 through full-stack C-V2X PC5 Mode 4 co-simulation. The evaluation covered 24 scenarios spanning six traffic levels-of-service with line-of-sight and non-line-of-sight propagation. PDR and end-to-end latency were evaluated at a roadside unit receiver. Dilithium-2 and SPHINCS+ exceeded the available transport-block capacity, whereas Falcon-512 remained physically feasible. Falcon-512 maintained mean latency near 52 ms and 95th-percentile latency within 97-98 ms, but met the 90% packet-delivery threshold only at traffic level-of-service A, under line-of-sight propagation. ECDSA met the threshold through traffic level-of-service C. Neither algorithm met the threshold under non-line-of-sight propagation. The study provides a standards-grounded cross-layer evaluation that identifies both algorithm feasibility and traffic-dependent deployment boundaries for post-quantum signatures on C-V2X Mode 4 sidelink. The results show that spectrum efficiency, rather than cryptographic computation time, is the primary deployment constraint. They support standards development concerning payload structure, resource allocation, certificate transmission, and migration strategies for quantum-resistant vehicular communication.

cs.PF

Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation

Autonomous vehicles (AVs) depend on Global Navigation Satellite Systems (GNSS) for localization and navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first Vision-Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.

cs.CV

AI-Driven Post-Quantum Cryptography for Cyber-Resilient V2X Communication in Transportation Cyber-Physical Systems

Transportation Cyber-Physical Systems (TCPS) integrate physical elements, such as transportation infrastructure and vehicles, with cyber elements via advanced communication technologies, allowing them to interact seamlessly. This integration enhances the efficiency, safety, and sustainability of transportation systems. TCPS rely heavily on cryptographic security to protect sensitive information transmitted between vehicles, transportation infrastructure, and other entities within the transportation ecosystem, ensuring data integrity, confidentiality, and authenticity. Traditional cryptographic methods have been employed to secure TCPS communications, but the advent of quantum computing presents a significant threat to these existing security measures. Therefore, integrating Post-Quantum Cryptography (PQC) into TCPS is essential to maintain secure and resilient communications. While PQC offers a promising approach to developing cryptographic algorithms resistant to quantum attacks, artificial intelligence (AI) can enhance PQC by optimizing algorithm selection, resource allocation, and adapting to evolving threats in real-time. AI-driven PQC approaches can improve the efficiency and effectiveness of PQC implementations, ensuring robust security without compromising system performance. This chapter introduces TCPS communication protocols, discusses the vulnerabilities of corresponding communications to cyber-attacks, and explores the limitations of existing cryptographic methods in the quantum era. By examining how AI can strengthen PQC solutions, the chapter presents cyber-resilient communication strategies for TCPS.

cs.CR

Security Vulnerabilities in Software Supply Chain for Autonomous Vehicles

The interest in autonomous vehicles (AVs) for critical missions, including transportation, rescue, surveillance, reconnaissance, and mapping, is growing rapidly due to their significant safety and mobility benefits. AVs consist of complex software systems that leverage artificial intelligence (AI), sensor fusion algorithms, and real-time data processing. Additionally, AVs are becoming increasingly reliant on open-source software supply chains, such as open-source packages, third-party software components, AI models, and third-party datasets. Software security best practices in the automotive sector are often an afterthought for developers. Thus, significant cybersecurity risks exist in the software supply chain of AVs, particularly when secure software development practices are not rigorously implemented. For example, Upstream's 2024 Automotive Cybersecurity Report states that 49.5% of cyberattacks in the automotive sector are related to exploiting security vulnerabilities in software systems. In this chapter, we analyze security vulnerabilities in open-source software components in AVs. We utilize static analyzers on popular open-source AV software, such as Autoware, Apollo, and openpilot. Specifically, this chapter covers: (1) prevalent software security vulnerabilities of AVs; and (2) a comparison of static analyzer outputs for different open-source AV repositories. The goal is to inform researchers, practitioners, and policymakers about the existing security flaws in the commonplace open-source software ecosystem in the AV domain. The findings would emphasize the necessity of security best practices earlier in the software development lifecycle to reduce cybersecurity risks, thereby ensuring system reliability, safeguarding user data, and maintaining public trust in an increasingly automated world.

cs.CR

End-to-End Co-Simulation Testbed for Cybersecurity Research and Development in Intelligent Transportation Systems

Intelligent Transportation Systems (ITS) have been widely deployed across major metropolitan regions worldwide to improve roadway safety, optimize traffic flow, and reduce environmental impacts. These systems integrate advanced sensors, communication networks, and data analytics to enable real-time traffic monitoring, adaptive signal control, and predictive maintenance. However, such integration significantly broadens the ITS attack surface, exposing critical infrastructures to cyber threats that jeopardize safety, data integrity, and operational resilience. Ensuring robust cybersecurity is therefore essential, yet comprehensive vulnerability assessments, threat modeling, and mitigation validations are often cost-prohibitive and time-intensive when applied to large-scale, heterogeneous transportation systems. Simulation platforms offer a cost-effective and repeatable means for cybersecurity evaluation, and the simulation platform should encompass the full range of ITS dimensions - mobility, sensing, networking, and applications. This chapter discusses an integrated co-simulation testbed that links CARLA for 3D environment and sensor modeling, SUMO for microscopic traffic simulation and control, and OMNeT++ for V2X communication simulation. The co-simulation testbed enables end-to-end experimentation, vulnerability identification, and mitigation benchmarking, providing practical insights for developing secure, efficient, and resilient ITS infrastructures. To illustrate its capabilities, the chapter incorporates a case study on a C-V2X proactive safety alert system enhanced with post-quantum cryptography, highlighting the role of the testbed in advancing secure and resilient ITS infrastructures.

cs.CR

Infrastructure-enabled risk assessment of hazardous road conditions on rural roads during inclement weather

Rural roadways often expose Commercial Motor Vehicle (CMV) drivers to hazardous conditions, such as heavy fog, rain, snow, black ice, and flash floods, many of which remain unreported in real time. This lack of timely information, coupled with limited infrastructure in rural areas, significantly increases the risk of crashes. Although various sensing technologies exist to monitor individual hazards like low visibility or surface friction, they rarely assess the combined driving risk posed by multiple simultaneous hazards, nor do they provide actionable recommendations such as safe advisory speeds. To address this critical gap, in this study, we present a roadway hazard risk assessment framework that provides an approach to quantify the probability and severity of crash occurrences due to specific roadway hazards. To evaluate this framework, we presented a case study by constructing a synthetic "year-long" dataset that encompasses every possible pairing of road surface and visibility conditions. Our analysis confirms that the combined ProbabilitySeverity scoring yields a coherent, stepwise risk profile across all hazard scenarios. These results validate the practicality of our risk assessment approach and provide a foundation for deploying graduated safety measures in real-world roadway operations.

cs.CE

Quantum-Classical Hybrid Framework for Zero-Day Time-Push GNSS Spoofing Detection

Global Navigation Satellite Systems (GNSS) are critical for Positioning, Navigation, and Timing (PNT) applications. However, GNSS are highly vulnerable to spoofing attacks, where adversaries transmit counterfeit signals to mislead receivers. Such attacks can lead to severe consequences, including misdirected navigation, compromised data integrity, and operational disruptions. Most existing spoofing detection methods depend on supervised learning techniques and struggle to detect novel, evolved, and unseen attacks. To overcome this limitation, we develop a zero-day spoofing detection method using a Hybrid Quantum-Classical Autoencoder (HQC-AE), trained solely on authentic GNSS signals without exposure to spoofed data. By leveraging features extracted during the tracking stage, our method enables proactive detection before PNT solutions are computed. We focus on spoofing detection in static GNSS receivers, which are particularly susceptible to time-push spoofing attacks, where attackers manipulate timing information to induce incorrect time computations at the receiver. We evaluate our model against different unseen time-push spoofing attack scenarios: simplistic, intermediate, and sophisticated. Our analysis demonstrates that the HQC-AE consistently outperforms its classical counterpart, traditional supervised learning-based models, and existing unsupervised learning-based methods in detecting zero-day, unseen GNSS time-push spoofing attacks, achieving an average detection accuracy of 97.71% with an average false negative rate of 0.62% (when an attack occurs but is not detected). For sophisticated spoofing attacks, the HQC-AE attains an accuracy of 98.23% with a false negative rate of 1.85%. These findings highlight the effectiveness of our method in proactively detecting zero-day GNSS time-push spoofing attacks across various stationary GNSS receiver platforms.

cs.LG

Grid2Guide: A* Enabled Small Language Model for Indoor Navigation

Reliable indoor navigation remains a significant challenge in complex environments, particularly where external positioning signals and dedicated infrastructures are unavailable. This research presents Grid2Guide, a hybrid navigation framework that combines the A* search algorithm with a Small Language Model (SLM) to generate clear, human-readable route instructions. The framework first conducts a binary occupancy matrix from a given indoor map. Using this matrix, the A* algorithm computes the optimal path between origin and destination, producing concise textual navigation steps. These steps are then transformed into natural language instructions by the SLM, enhancing interpretability for end users. Experimental evaluations across various indoor scenarios demonstrate the method's effectiveness in producing accurate and timely navigation guidance. The results validate the proposed approach as a lightweight, infrastructure-free solution for real-time indoor navigation support.

cs.LG

Vision-Based Localization and LLM-based Navigation for Indoor Environments

Indoor navigation remains a complex challenge due to the absence of reliable GPS signals and the architectural intricacies of large enclosed environments. This study presents an indoor localization and navigation approach that integrates vision-based localization with large language model (LLM)-based navigation. The localization system utilizes a ResNet-50 convolutional neural network fine-tuned through a two-stage process to identify the user's position using smartphone camera input. To complement localization, the navigation module employs an LLM, guided by a carefully crafted system prompt, to interpret preprocessed floor plan images and generate step-by-step directions. Experimental evaluation was conducted in a realistic office corridor with repetitive features and limited visibility to test localization robustness. The model achieved high confidence and an accuracy of 96% across all tested waypoints, even under constrained viewing conditions and short-duration queries. Navigation tests using ChatGPT on real building floor maps yielded an average instruction accuracy of 75%, with observed limitations in zero-shot reasoning and inference time. This research demonstrates the potential for scalable, infrastructure-free indoor navigation using off-the-shelf cameras and publicly available floor plans, particularly in resource-constrained settings like hospitals, airports, and educational institutions.

cs.LG

Lessons Learned from the Real-World Deployment of Multi-Sensor Fusion for Proactive Work Zone Safety Application

Proactive safety systems that anticipate and mitigate traffic risks before incidents occur are increasingly recognized as essential for improving work zone safety. Unlike traditional reactive methods, these systems rely on real-time sensing, trajectory prediction, and intelligent infrastructure to detect potential hazards. Existing simulation-based studies often overlook, and real-world deployment studies rarely discuss the practical challenges associated with deploying such systems in operational settings, particularly those involving roadside infrastructure and multi-sensor integration and fusion. This study addresses that gap by presenting deployment insights and technical lessons learned from a real-world implementation of a multi-sensor safety system at an active bridge repair work zone along the N-2/US-75 corridor in Lincoln, Nebraska. The deployed system combines LiDAR, radar, and camera sensors with an edge computing platform to support multi-modal object tracking, trajectory fusion, and real-time analytics. Specifically, this study presents key lessons learned across three critical stages of deployment: (1) sensor selection and placement, (2) sensor calibration, system integration, and validation, and (3) sensor fusion. Additionally, we propose a predictive digital twin framework that leverages fused trajectory data for early conflict detection and real-time warning generation, enabling proactive safety interventions.

cs.OH

OpenCAMS: An Open-Source Connected and Automated Mobility Co-Simulation Platform for Advancing Next-Generation Intelligent Transportation Systems Research

We introduce OpenCAMS (Open-Source Connected and Automated Mobility Co-Simulation Platform), an open-source, synchronized, and extensible co-simulation framework that tightly couples three best-in-class simulation tools: (i) SUMO, (ii) CARLA, and (iii) OMNeT++. OpenCAMS is designed to support advanced research in transportation safety, mobility, and cybersecurity by combining the strengths of each simulation domain. Specifically, SUMO provides large-scale, microscopic traffic modeling; CARLA offers high-fidelity 3D perception, vehicle dynamics, and control simulation; and OMNeT++ enables modular, event-driven network communication, such as cellular vehicle-to-everything (C-V2X). OpenCAMS employs a time-synchronized, bidirectional coupling architecture that ensures coherent simulation progression across traffic, perception, and communication domains while preserving modularity and reproducibility. For example, CARLA can simulate and render a subset of vehicles that require detailed sensor emulation and control logic; SUMO orchestrates network-wide traffic flow, vehicle routing, and traffic signal management; and OMNeT++ dynamically maps communication nodes to both mobile entities (e.g., vehicles) and static entities (e.g., roadside units) to enable C-V2X communication. While these three simulators form the foundational core of OpenCAMS, the platform is designed to be expandable and future-proof, allowing additional simulators to be integrated on top of this core without requiring fundamental changes to the system architecture. The OpenCAMS platform is fully open-source and publicly available through its GitHub repository https://github.com/minhaj6/carla-sumo-omnetpp-cosim, providing the research community with an accessible, flexible, and collaborative environment for advancing next-generation intelligent transportation systems.

cs.SE

Experimental Validation of Sensor Fusion-based GNSS Spoofing Attack Detection Framework for Autonomous Vehicles

In this paper, we validate the performance of the a sensor fusion-based Global Navigation Satellite System (GNSS) spoofing attack detection framework for Autonomous Vehicles (AVs). To collect data, a vehicle equipped with a GNSS receiver, along with Inertial Measurement Unit (IMU) is used. The detection framework incorporates two strategies: The first strategy involves comparing the predicted location shift, which is the distance traveled between two consecutive timestamps, with the inertial sensor-based location shift. For this purpose, data from low-cost in-vehicle inertial sensors such as the accelerometer and gyroscope sensor are fused and fed into a long short-term memory (LSTM) neural network. The second strategy employs a Random-Forest supervised machine learning model to detect and classify turns, distinguishing between left and right turns using the output from the steering angle sensor. In experiments, two types of spoofing attack models: turn-by-turn and wrong turn are simulated. These spoofing attacks are modeled as SQL injection attacks, where, upon successful implementation, the navigation system perceives injected spoofed location information as legitimate while being unable to detect legitimate GNSS signals. Importantly, the IMU data remains uncompromised throughout the spoofing attack. To test the effectiveness of the detection framework, experiments are conducted in Tuscaloosa, AL, mimicking urban road structures. The results demonstrate the framework's ability to detect various sophisticated GNSS spoofing attacks, even including slow position drifting attacks. Overall, the experimental results showcase the robustness and efficacy of the sensor fusion-based spoofing attack detection approach in safeguarding AVs against GNSS spoofing threats.

cs.CR

Unveiling the Stealthy Threat: Analyzing Slow Drift GPS Spoofing Attacks for Autonomous Vehicles in Urban Environments and Enabling the Resilience

Autonomous vehicles (AVs) rely on the Global Positioning System (GPS) or Global Navigation Satellite Systems (GNSS) for precise (Positioning, Navigation, and Timing) PNT solutions. However, the vulnerability of GPS signals to intentional and unintended threats due to their lack of encryption and weak signal strength poses serious risks, thereby reducing the reliability of AVs. GPS spoofing is a complex and damaging attack that deceives AVs by altering GPS receivers to calculate false position and tracking information leading to misdirection. This study explores a stealthy slow drift GPS spoofing attack, replicating the victim AV's satellite reception pattern while changing pseudo ranges to deceive the AV, particularly during turns. The attack is designed to gradually deviate from the correct route, making real-time detection challenging and jeopardizing user safety. We present a system and study methodology for constructing covert spoofing attacks on AVs, investigating the correlation between original and spoofed pseudo ranges to create effective defenses. By closely following the victim vehicle and using the same satellite signals, the attacker executes the attack precisely. Changing the pseudo ranges confuses the AV, leading it to incorrect destinations while remaining oblivious to the manipulation. The gradual deviation from the actual route further conceals the attack, hindering its swift identification. The experiments showcase a robust correlation between the original and spoofed pseudo ranges, with R square values varying between 0.99 and 1. This strong correlation facilitates effective evaluation and mitigation of spoofing signals.

cs.CR

Digital Twin Technology Enabled Proactive Safety Application for Vulnerable Road Users: A Real-World Case Study

While measures, such as traffic calming and advance driver assistance systems, can improve safety for Vulnerable Road Users (VRUs), their effectiveness ultimately relies on the responsible behavior of drivers and pedestrians who must adhere to traffic rules or take appropriate actions. However, these measures offer no solution in scenarios where a collision becomes imminent, leaving no time for warning or corrective actions. Recently, connected vehicle technology has introduced warning services that can alert drivers and VRUs about potential collisions. Nevertheless, there is still a significant gap in the system's ability to predict collisions in advance. The objective of this study is to utilize Digital Twin (DT) technology to enable a proactive safety alert system for VRUs. A pedestrian-vehicle trajectory prediction model has been developed using the Encoder-Decoder Long Short-Term Memory (LSTM) architecture to predict future trajectories of pedestrians and vehicles. Subsequently, parallel evaluation of all potential future safety-critical scenarios is carried out. Three Encoder-Decoder LSTM models, namely pedestrian-LSTM, vehicle-through-LSTM, and vehicle-left-turn-LSTM, are trained and validated using field-collected data, achieving corresponding root mean square errors (RMSE) of 0.049, 1.175, and 0.355 meters, respectively. A real-world case study has been conducted where a pedestrian crosses a road, and vehicles have the option to proceed through or left-turn, to evaluate the efficacy of DT-enabled proactive safety alert systems. Experimental results confirm that DT-enabled safety alert systems were succesfully able to detect potential crashes and proactively generate safety alerts to reduce potential crash risk.

cs.RO