A Counterfactual Diagnostic Framework for Explaining KS Deterioration in Credit Risk Model Validation

Abstract

The Kolmogorov-Smirnov (KS) statistic is widely used in credit risk model monitoring and validation to assess discriminatory power. In practice, a material decline in KS often triggers governance review and requires validation teams to identify the breach source and the potential business risk. However, such diagnosis is frequently conducted on an ad hoc basis, relying on the judgment of individual validators rather than a standardized analytical framework. This paper proposes a counterfactual diagnostic framework for explaining KS deterioration in credit risk model validation. The framework sequentially attributes observed KS decline to sampling variability, portfolio composition change, covariate shift, and residual deterioration consistent with model drift, with explicit gateway conditions governing escalation at each stage. Simulation experiments demonstrate that the proposed approach provides more interpretable and governance-relevant explanations than threshold-based review alone, and contributes to more consistent, transparent, and defensible performance-breach assessment in credit risk model validation.