arXiv · 2611.08714
Patching the Blind Spots: Reversible LLM Rule Overlay for Frozen Fraud Detectors
Abstract
Gradient-boosted trees dominate industrial tabular fraud classification, but even a well-trained model has blind spots, and retraining a deployed model is not always safe. We bring LLM reasoning to patching such a model without modifying it: LLMs read its tree digest and SHAP attributions, reason about the fraud it misses, and propose rules anchored on features it underweights, screened and calibrated into a reversible log-lift overlay at the production cut-off. Single-LLM benefit is high-variance and inverts between datasets, so we pool three diverse backbones by logit averaging, trading peak gain for a bounded worst case. On an experimental detector trained on sampled, selection-biased real-world e-commerce data, tuned continued training regresses, whereas the overlay gains on average +2.26% recall at equal precision ($Δ$R@P) and +1.60% precision at equal recall ($Δ$P@R). Replayed offline on a legacy production detector, the overlay, calibrated on one month and frozen, stays more precise at equal recall for seven months (+5.64% $Δ$P@R). On seven bias-shift variants of a public benchmark it raises $Δ$P@R on all and keeps $Δ$R@P near neutral. LLM reasoning over a model's diagnostics can improve a deployed tabular classifier through readable predicates naming the fraud they cover.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Bo Qu, Zhurong Wang, Frank Zahradnik, Ming Ouyang. 2026-11-11. Patching the Blind Spots: Reversible LLM Rule Overlay for Frozen Fraud Detectors. https://arxiv.science/abs/2611.08714
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.