arXiv subjects

Budi Arief

Publications and source records attributed to Budi Arief.

8 recordsLinked to original sources

arXiv2026

APIOT: Autonomous Vulnerability Management Across Bare-Metal Industrial OT Networks

Operational technology (OT) devices run safety-critical physical processes, yet their security testing remains manual and expert-intensive. Autonomous security agents could ease this burden, but bare-metal OT devices are difficult targets because they lack shells, filesystems, and named exploit abstractions, requiring operation through protocol fields and observable device state. We present APIOT (Autonomous Purple-teaming for Industrial OT), a large language model (LLM) agent framework that completes a discovery -> exploitation -> network-level mitigation -> verification workflow without human intervention. We evaluate APIOT in 360 runs on vulnerable firmware and simulators for Zephyr RTOS, spanning five LLMs, three topologies, two impairment levels, and prompt, tooling and two ablations. APIOT completed 90% of primary missions (63/70 across protocols and topologies) and 85% of 150 cross-model runs (128/150) - feasible across models, with substantial variation. Named exploit tools (e.g., a canned CoAP option-overflow) were faster, but protocol primitives (the agent builds the fields itself) produced more reusable mitigations that blocked unseen payload variants. Runtime oversight improved reliability: success rose from 27/30 to 29/30 with rule-based guards and 30/30 when a second LLM added advisory steering. To our knowledge, this is the first end-to-end autonomous attack-remediation demonstration on bare-metal industrial devices, establishing controlled feasibility - not zero-day discovery or production-OT transfer.

cs.CR↗

arXiv2026

Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis

Malware analysis demands rapid interpretation of complex detonation reports spanning filesystem, network, and process behaviours. While large language models (LLMs) demonstrate impressive capabilities for technical artifact interpretation, the opacity and escalating API costs of closed-weight frontier models motivate exploration of open-weight alternatives. However, many open-weight models are large, demanding significant compute resources and incurring non-trivial hosting costs that place them beyond reach for resource-constrained deployments. This paper investigates whether orchestrated ensembles of small language models (SLMs) can match or exceed single LLM performance on structured questions about malware detonation reports. We established baselines by testing eleven open-weight SLMs, three cyber security pre-trained models, and six frontier LLMs on Meta's CyberSecEval Malware Analysis benchmark. We then designed and evaluated four orchestration architectures: (i) a multi-agent pipeline that decomposes analysis into structured evidence-collection and reasoning stages, (ii) an adversarial debate framework in which two agents iteratively critique each other's reasoning, (iii) a hierarchical consultation system that pairs a general-purpose SLM with a cyber-specialised expert model, and (iv) a hybrid architecture that combines evidence-grounded pipelines with adversarial debate reasoning. The hybrid system (Qwen3-4B with Foundation-Sec-8B) achieved 35.30% overall accuracy, exceeding the strongest cyber-specialised baseline (22.54%) and the strongest ungrounded frontier baseline (34.77%); when given the same evidence pipeline, grounded Gemini remained the strongest configuration at 38.22%. These findings show that evidence-grounded orchestration can substantially improve the performance of collaborative SLMs for supporting interpretation of malware detonation reports.

cs.CR↗

arXiv2026

Agentic Knowledge Distillation: Autonomous Training of Small Language Models for SMS Threat Detection

SMS-based phishing (smishing) attacks have surged, yet training effective on-device detectors requires labelled threat data that quickly becomes outdated. To deal with this issue, we present Agentic Knowledge Distillation, which consists of a powerful LLM acts as an autonomous teacher that fine-tunes a smaller student SLM, deployable for security tasks without human intervention. The teacher LLM autonomously generates synthetic data and iteratively refines a smaller on-device student model until performance plateaus. We compare four LLMs in this teacher role (Claude Opus 4.5, GPT 5.2 Codex, Gemini 3 Pro, and DeepSeek V3.2) on SMS spam/smishing detection with two student SLMs (Qwen2.5-0.5B and SmolLM2-135M). Our results show that performance varies substantially depending on the teacher LLM, with the best configuration achieving 94.31% accuracy and 96.25% recall. We also compare against a Direct Preference Optimisation (DPO) baseline that uses the same synthetic knowledge and LoRA setup but without iterative feedback or targeted refinement; agentic knowledge distillation substantially outperforms it (e.g. 86-94% vs 50-80% accuracy), showing that closed-loop feedback and targeted refinement are critical. These findings demonstrate that agentic knowledge distillation can rapidly yield effective security classifiers for edge deployment, but outcomes depend strongly on which teacher LLM is used.

cs.CR↗

arXiv2025

Analysing Safety Risks in LLMs Fine-Tuned with Pseudo-Malicious Cyber Security Data

Large language models (LLMs) have been used in many application domains, including cyber security. The application of LLMs in the cyber security domain presents significant opportunities, such as for enhancing threat analysis and malware detection, but it can also introduce critical risks and safety concerns, including potential personal data leakage and automated generation of new malware. Building on recent findings that fine-tuning LLMs with pseudo-malicious cyber security data significantly compromises their safety, this paper presents a comprehensive validation and extension of these safety risks using a different evaluation framework. We employ the garak red teaming framework with the OWASP Top 10 for LLM Applications to assess four open-source LLMs: Mistral 7B, Llama 3 8B, Gemma 2 9B, and DeepSeek R1 8B. Our evaluation confirms and extends previous findings, showing that fine-tuning reduces safety resilience across all tested LLMs (e.g., the failure rate of Mistral 7B against prompt injection increases from 9.1% to 68.7%). We further propose and evaluate a novel safety alignment approach that carefully rewords instruction-response pairs to include explicit safety precautions and ethical considerations. This work validates previous safety concerns through independent evaluation and introduces new methods for mitigating these risks, contributing towards the development of secure, trustworthy, and ethically aligned LLMs. This approach demonstrates that it is possible to maintain or even improve model safety while preserving technical utility, offering a practical path towards developing safer fine-tuning methodologies.

cs.CR↗

arXiv2025

CyberLLMInstruct: A Pseudo-malicious Dataset Revealing Safety-performance Trade-offs in Cyber Security LLM Fine-tuning

The integration of large language models (LLMs) into cyber security applications presents both opportunities and critical safety risks. We introduce CyberLLMInstruct, a dataset of 54,928 pseudo-malicious instruction-response pairs spanning cyber security tasks including malware analysis, phishing simulations, and zero-day vulnerabilities. Our comprehensive evaluation using seven open-source LLMs reveals a critical trade-off: while fine-tuning improves cyber security task performance (achieving up to 92.50% accuracy on CyberMetric), it severely compromises safety resilience across all tested models and attack vectors (e.g., Llama 3.1 8B's security score against prompt injection drops from 0.95 to 0.15). The dataset incorporates diverse sources including CTF challenges, academic papers, industry reports, and CVE databases to ensure comprehensive coverage of cyber security domains. Our findings highlight the unique challenges of securing LLMs in adversarial domains and establish the critical need for developing fine-tuning methodologies that balance performance gains with safety preservation in security-sensitive domains.

cs.CR↗

arXiv2024

Privacy Threats and Countermeasures in Federated Learning for Internet of Things: A Systematic Review

Federated Learning (FL) in the Internet of Things (IoT) environments can enhance machine learning by utilising decentralised data, but at the same time, it might introduce significant privacy and security concerns due to the constrained nature of IoT devices. This represents a research challenge that we aim to address in this paper. We systematically analysed recent literature to identify privacy threats in FL within IoT environments, and evaluate the defensive measures that can be employed to mitigate these threats. Using a Systematic Literature Review (SLR) approach, we searched five publication databases (Scopus, IEEE Xplore, Wiley, ACM, and Science Direct), collating relevant papers published between 2017 and April 2024, a period which spans from the introduction of FL until now. Guided by the PRISMA protocol, we selected 49 papers to focus our systematic review on. We analysed these papers, paying special attention to the privacy threats and defensive measures -- specifically within the context of IoT -- using inclusion and exclusion criteria tailored to highlight recent advances and critical insights. We identified various privacy threats, including inference attacks, poisoning attacks, and eavesdropping, along with defensive measures such as Differential Privacy and Secure Multi-Party Computation. These defences were evaluated for their effectiveness in protecting privacy without compromising the functional integrity of FL in IoT settings. Our review underscores the necessity for robust and efficient privacy-preserving strategies tailored for IoT environments. Notably, there is a need for strategies against replay, evasion, and model stealing attacks. Exploring lightweight defensive measures and emerging technologies such as blockchain may help improve the privacy of FL in IoT, leading to the creation of FL models that can operate under variable network conditions.

cs.CR↗

arXiv2023

The Use of Public Data and Free Tools in National CSIRTs' Operational Practices: A Systematic Literature Review

Many CSIRTs, including national CSIRTs, routinely use public data, including open-source intelligence (OSINT) and free tools, which include open-source tools in their work. However, we observed a lack of public information and systematic discussions regarding how national CSIRTs use and perceive public data and free tools in their operational practices. Therefore, this paper provides a systematic literature review (SLR) to comprehensively understand how national CSIRTs use and perceive public data and free tools in facilitating incident responses in operations. Our SLR method followed a three-stage approach: 1) a systematic search to identify relevant publications from websites of pertinent CSIRT organisations, 2) a conventional SLR into the research literature, and 3) synthesise data from stages one and two to answer the research questions. In the first stage, we searched the websites of 100 national CSIRTs and 11 cross-CSIRT organisations to identify relevant information about national CSIRTs. In the second stage, we searched a scientific database (Scopus) to identify relevant research papers. Our primary finding from the SLR is that most discussions concerning public data and free tools by national CSIRTs are incomplete, ad hoc, or fragmented. We discovered a lack of discussions on how the staff of national CSIRTs perceive the usefulness of public data and free tools to facilitate incident responses. Such gaps can prevent us from understanding how national CSIRTs can benefit from public data and free tools and how other organisations, individuals and researchers can help by providing such data and tools to improve national CSIRTs' operation. These findings call for more empirical research on how national CSIRTs use and perceive public data and free tools to improve the overall incident responses at national CSIRTs and other incident response organisations.

cs.DL↗

arXiv2020

Position paper: A systematic framework for categorising IoT device fingerprinting mechanisms

The popularity of the Internet of Things (IoT) devices makes it increasingly important to be able to fingerprint them, for example in order to detect if there are misbehaving or even malicious IoT devices in one's network. The aim of this paper is to provide a systematic categorisation of machine learning augmented techniques that can be used for fingerprinting IoT devices. This can serve as a baseline for comparing various IoT fingerprinting mechanisms, so that network administrators can choose one or more mechanisms that are appropriate for monitoring and maintaining their network. We carried out an extensive literature review of existing papers on fingerprinting IoT devices -- paying close attention to those with machine learning features. This is followed by an extraction of important and comparable features among the mechanisms outlined in those papers. As a result, we came up with a key set of terminologies that are relevant both in the fingerprinting context and in the IoT domain. This enabled us to construct a framework called IDWork, which can be used for categorising existing IoT fingerprinting mechanisms in a way that will facilitate a coherent and fair comparison of these mechanisms. We found that the majority of the IoT fingerprinting mechanisms take a passive approach -- mainly through network sniffing -- instead of being intrusive and interactive with the device of interest. Additionally, a significant number of the surveyed mechanisms employ both static and dynamic approaches, in order to benefit from complementary features that can be more robust against certain attacks such as spoofing and replay attacks.

cs.NI↗