A Proactive Insider Threat Management Framework Using Explainable Machine Learning

Abstract

Over the years, the technological landscape has evolved, reshaping the security posture of organisations and increasing their exposure to cybersecurity threats, many originating from within. Insider threats remain a major challenge, particularly in sectors where cybersecurity infrastructure, expertise, and regulations are still developing. This study proposes the Insider Threat Explainable Machine Learning (IT-XML) framework, which integrates the Cross-Industry Standard Process for Data Mining (CRISP-DM) with Hidden Markov Models (HMM) to enhance proactive insider threat management and decision-making. A quantitative approach is adopted using an online questionnaire to assess employees' knowledge of insider threat patterns, access control, privacy practices, and existing policies across three large data-sensitive organisations. The IT-XML framework provides assessment capabilities through survey-based data, HMM-driven pattern recognition for security maturity classification, and evidence-based recommendations for proactive threat mitigation. The framework classified all organisations at the developing security maturity level with 97-98% confidence and achieved a classification accuracy of 91.7%, identifying audit log access limits as the most critical control. Random Forest analysis highlighted vendor breach notifications (0.081) and regular audit log reviews (0.052) as key determinants of resilience. Explainability methods such as SHAP and LIME improved model transparency and interpretability, demonstrating the framework's potential to strengthen insider threat management practices.