arXiv · 2611.20695
IndustrialKnowEval: A Benchmark for Evaluating Multi-Level Industrial Knowledge in LLMs
Abstract
Large language models (LLMs) are increasingly explored for industrial tasks, yet few benchmarks evaluate their mastery of industrial knowledge across different types of cognitive demand. We introduce IndustrialKnowEval (IndKnow), a benchmark for industrial knowledge and engineering question answering. It contains 43,282 samples covering 21 task types across seven industrial sectors, including aerospace, electric power, and metallurgy, organized into four levels of cognitive demand: memory, comprehension, reasoning, and engineering application. Enterprise-based domain specialists and experienced researchers participate in all stages of dataset construction and quality control. We conduct a series of experiments on 18 representative LLMs, with domain experts deeply involved in the evaluation. Building upon IndKnow, we propose the Industrial Cognitive Capability Gap (ICCG) metric to characterize the performance differential between foundational industrial knowledge and higher-order reasoning and application, and we analyze it alongside absolute performance to provide a more complete view of model capabilities. We anticipate that IndustrialKnowEval will support the benchmarking of LLMs in industrial scenarios and promote the development of robust industrial intelligence. Code is available at: https://github.com/J4cksonLee/IndKnow
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Haiteng Wang, Haoze Li, Jingwei Guo, Jingbo Zhou, Si Liu, Lei Ren. 2026-11-16. IndustrialKnowEval: A Benchmark for Evaluating Multi-Level Industrial Knowledge in LLMs. https://arxiv.science/abs/2611.20695
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.