arXiv · 2611.18575
A Survey of Image-to-Code Generation for Software Engineering
Abstract
A large fraction of software intent is expressed visually before it is implemented. A designer hands a screenshot or a Figma mockup to a front-end engineer, an architect sketches a UML diagram or a flowchart, and a bug report ships with a picture of what went wrong. Turning such visual artifacts directly into executable software has therefore become an active problem at the intersection of computer vision and software engineering. The present survey organizes the literature on image-to-code generation for software engineering, defined as the family of methods whose input contains a visual modality and whose output is software an engineer would otherwise write by hand. We deliberately scope the survey to software-engineering code, namely user-interface and front-end code, code recovered from software diagrams such as UML and flowcharts, program repair and test code driven by visual signals, and general software-code targets, and we exclude adjacent code-generation domains whose home communities lie elsewhere, such as chart plotting scripts, vector-graphics programs, hardware description languages, and robot policies. Rather than cataloguing systems in isolation, we introduce a unified processing pipeline of visual understanding, component recognition, layout reasoning, code planning, code generation, and verification, and organize the survey around its stages. We trace five overlapping eras from rule-based reverse engineering to agentic generation, survey each pipeline stage and each software-engineering task in depth grounded in what individual systems actually do, consolidate the datasets, benchmarks, and metrics, and lay out the open challenges of design-system generalization, long-horizon and interactive generation, cross-framework deployment, accessibility, hallucinated components, and unreliable evaluation.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Licheng Zhang, Xihong Wu, Dingsheng Luo, Siew-Kei Lam. 2026-07-28. A Survey of Image-to-Code Generation for Software Engineering. https://arxiv.science/abs/2611.18575
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.