arXiv Science⌕ Search

arXiv subjects

Alexandra Newcomb

Publications and source records attributed to Alexandra Newcomb.

2 recordsLinked to original sources

Automatic Translation of Unstructured Requirements into Linear Temporal Logic through Large Language Models

Automatically translating unstructured natural language requirements into formal specifications remains a challenge in requirements engineering and formal methods, particularly for safety- and mission-critical systems whose verification depends on mathematically precise specifications. This paper evaluates whether contemporary off-the-shelf Large Language Models (LLMs) can help bridge this gap by generating Linear Temporal Logic (LTL) formulas directly from unstructured requirements. The study examines six modern LLMs using a few-shot prompting strategy on a heterogeneous benchmark of 15 structurally varied requirements. Five independent generations were collected for each requirement-model pair, yielding 450 candidate LTL formulas in total. Performance was assessed through manual semantic evaluation, pass@k for k in {1, 3, 5}, and a self-consistency measure capturing syntactic reproducibility across stochastic trials. The results indicate that current general-purpose LLMs can achieve practically significant performance on the unstructured NL-to-LTL task without task-specific fine-tuning. The study also considers understandability for non-experts by pairing generated formulas with model-produced natural language explanations and discussing the complementary use of timeline-based LTL visualization. The findings suggest that modern LLMs are becoming viable front-end assistants for semi-automated formalization workflows.

cs.SE↗

Formal Modeling in Software Engineering with LLMs: A Systematic Literature Review

Formal verification provides strong correctness guarantees for software systems but remains hindered by the substantial expertise and manual effort required to construct and maintain formal models. The recent rise of Large Language Models (LLMs) has prompted growing interest in whether their natural language understanding and generation capabilities can be leveraged to automate labor-intensive modeling activities. This paper presents a Systematic Literature Review (SLR) examining how LLMs have been applied to automate model checking and the generation of formal and semi-formal models within software engineering. The review follows established SLR methodologies and a PICOC-driven search strategy across four major digital libraries. From an initial search of 1,193 articles, 34 peer-reviewed conference, journal, and workshop articles published between 2022 and April 2026 were selected and analyzed using a structured data extraction form. The review identifies six recurring trends in the literature, including the automated generation of formal models for model checking, LLM-assisted downstream formal verification, iterative and repair-based generation pipelines, and the generation of semi-formal UML and SysML artifacts whose underlying techniques are transferable to formal modeling contexts. Across these trends, LLMs are generally found to perform well on syntactic generation and repair tasks but struggle with semantic correctness, motivating widespread use of counterexample-guided repair loops, intermediate representations, and human-in-the-loop validation. Notably, this review finds that the use of LLMs specifically to automate model checking remains a largely unexplored area, with comparatively little research published on this narrower task.

cs.SE↗