arXiv Science⌕ Search

arXiv · 2611.18127

PRISM: A Post-Prediction Reliability Framework for Evidence-Guided Correction and Selective Abstention

Abstract

A model may produce a prediction with high confidence and still get the outcome wrong. This is a challenge in sequential applications, where the system evolves over time and deployment data may differ from the data used during training. To address this issue, we propose PRISM (Prediction Reliability and Intelligent Self-Correction Mechanism), a framework that evaluates prediction reliability after the initial output is generated. It estimates the risk of error and examines predictions identified as potentially unreliable. For these cases, PRISM analyses available signals, retrieves similar situations from historical data, and verifies whether an alternative prediction is supported by the evidence. It can then retain the original prediction, apply a verified correction, or abstain when the evidence is insufficient. We evaluate PRISM on second-innings match prediction in the Indian Premier League (IPL). Ball-by-ball records are converted into over-level observations to capture changes in match conditions throughout an innings. To preserve temporal order, earlier IPL seasons are used for training, IPL 2024 for validation, and IPL 2025-2026 for out-of-time testing. The dataset contains 8,665 over-level observations from 475 matches, with the test partition comprising 2,568 observations from 142 matches. In one experimental record, Logistic Regression, Random Forest, and HistGradientBoosting achieve accuracies of 65.2%, 68.4%, and 70.2%, respectively. The corresponding PRISM evaluation reports a selective accuracy of 82.4% at 88.5% coverage, with a Brier score of 0.126 and an expected calibration error (ECE) of 0.019.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Omm Swarup, Shrishail M. Hiremath. 2026-11-13. PRISM: A Post-Prediction Reliability Framework for Evidence-Guided Correction and Selective Abstention. https://arxiv.science/abs/2611.18127

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Flux: Optimal Scheduling of Optical Circuit Switches for LLM Training

Optical Circuit Switching (OCS) offers high bandwidth density and energy efficiency for LLM training, but incurs a non-negligible reconfiguration delay. Prior work typically schedules optical circuit switches independently of compute, using aggregate traffic demand to determine which circuits to provision and when. We argue that this separation creates a fundamental inefficiency: reconfigurations that ignore the compute timeline can stall communication, resulting in low circuit utilization and large buffer requirements. In this paper, we present Flux, a scheduler that optimally schedules optical circuit switches based on the structure of the entire workload. Flux remains effective across a wide range of switching speeds by reusing circuits and amortizing reconfiguration delay behind compute and communication. We show that Flux reduces training iteration time by up to $10\times$ and peak NIC buffer requirements by more than three orders of magnitude compared to traditional periodic schedulers.

cs.NI↗

HiGAI-Net: Hierarchical Encrypted Traffic Classification for Generative AI Services

The Radio Access Network (RAN) today does not have a reliable way to identify GenAI traffic (ChatGPT, Microsoft Copilot, or Google Gemini traffic) at the bearer level. TLS~1.3 moves SNI into the encrypted handshake record, and Encrypted ClientHello (ECH) removes it entirely from passive inspection; Gemini has partly shifted to QUIC where SNI visibility also depends on whether ECH is deployed. Without per-flow application identity, differentiated QoS for GenAI services is not possible. HiGAI-Net tackles this using only packet-header metadata, stacking three independently trained classifiers: a binary AI/Non-AI gate (98.5 % weighted F1), a per-service resolver across ChatGPT, Copilot, and Gemini (86.4 %), and a text-vs-voice/video mode classifier (61.0 %). The core innovation is a suite of sliding-window temporal features that track how DL/UL byte-ratios evolve within a session -- the per-quarter and per-half ratio trends, a trend differential $ΔR$, and a streaming regularity index. Tested on 44,492 labeled flows, gradient-boosted tree models consistently outperformed all deep-learning variants; a per-flow label cache brings amortised inference below 1~$μ$s per packet.

cs.NI↗

Do LLMs Understand Network Traces? A Comparative Study of Reasoning over Raw and Filtered Traffic Data

The effective delivery and monitoring of ICT services rely heavily on network traffic analysis. However, interpreting complex packet captures remains a challenging task that requires substantial technical expertise. Large language models (LLMs) offer promising opportunities to automate network traffic in- vestigation, yet their reliability in packet-level field extraction and flow-level sequence reconstruction remains an open research question. This study proposes a reproducible, question-answering- based evaluation methodology to assess the capabilities of LLMs in network trace analysis. Using real-world benign traffic from the CICIDS2017 dataset, we evaluate multiple models against a human-generated ground truth, comparing their performance on raw textual traffic dumps and protocol-filtered representations. Model responses are assessed through an automated LLM- as-a-judge pipeline, validated against human evaluations. The results demonstrate that protocol filtering significantly reduces prompt overhead while improving answer accuracy by reducing incorrect responses. These findings highlight the importance of protocol-aware traffic filtering as an effective optimization for improving the efficiency and reliability of LLM-assisted network management.

cs.NI↗