Multi-Agent Retrieval-Augmented Generation for Efficient Cloud Knowledge Base Search in Telecom SNOC Environment

Abstract

Telecom Service and Network Operations Centers (SNOCs) rely on large collections of cloud documents, including Standard Operating Procedures (SOPs), vendor technical manuals, incident reports, and configuration guides, to maintain uninterrupted network operations. During critical incidents, engineers must quickly retrieve accurate information, yet traditional keyword based and single stage retrieval approaches often struggle to provide precise results. This paper presents Athena for Cloud Knowledge Base, a fully offline, multi agent Retrieval Augmented Generation (RAG) framework designed for enterprise cloud document search in Vodafone Idea's SNOC environment. The system integrates dense retrieval using E5 Large V2 embeddings, BM25 sparse retrieval, and Knowledge Graph expansion within a LangGraph based orchestration framework. Retrieved candidates are fused using Weighted CombSUM, followed by cross encoder reranking and Maximal Marginal Relevance (MMR) to obtain a diverse and relevant evidence set. To improve answer reliability, the framework performs per chunk LLM evaluation with explicit attribution verification, assessing each MMR selected chunk independently before generating a response. Unsupported or weak evidence is discarded, and if no chunk satisfies the verification criteria, the system automatically evaluates multiple chunks together as a fallback. Experiments on a corpus of 4200 SNOC cloud documents containing 312000 indexed chunks show that the proposed approach achieves an MRR at 10 of 0.910 and an Exact Match (EM) score of 78.4 percent, outperforming single stage dense retrieval by 14.6 percentage points. The entire pipeline operates in a fully offline environment, satisfying enterprise data sovereignty requirements while delivering accurate and grounded responses for cloud document search.