Cross-Dataset Generalization of Glaucoma Detection CNNs: Effects of Operating Threshold, Multi-Source Training, and Color Normalization

Abstract

Deep learning models for glaucoma detection are usually tested on images from the same dataset used for training. That says little about how they will work at a new hospital or with a new camera. We fine-tuned five ImageNet-pretrained CNNs (ResNet50, DenseNet121, MobileNetV2, EfficientNet-B0, InceptionV3) with identical settings on one of three public fundus datasets (ACRIMA, ORIGA, RIM-ONE DL). Each was tested on a held-out part of its own dataset and on the other two datasets in full. We used three random seeds and report mean and SD over seeds. Mean in-domain AUC was 0.913 and cross-source AUC was 0.702, a drop of 0.211 (SD 0.030; 95% CI 0.135 to 0.286). After correction for multiple comparisons we found no evidence that any architecture did better, though three seeds give low power. At thresholds set on the source validation data, specificity fell from 0.870 to 0.506 and sensitivity from 0.831 to 0.726. Resetting the threshold on 10% of the target images raised specificity to 0.670 but lowered sensitivity to 0.609, and balanced accuracy rose only from 0.616 to 0.639. Training on two sources together did not help (held-out AUC 0.695 versus 0.702). LAB reference normalization (cross-source AUC 0.755) and strong color augmentation (0.754) gave similar gains, cutting the drop to about 0.16. Ben Graham normalization (0.733) and CLAHE (0.712) helped less, and gray-world (0.696) did not help. The two best methods raised specificity but lowered sensitivity. On G1020, used only for testing, AUC stayed near 0.5 in every condition, and networks trained within G1020 were also close to chance. Strong in-domain results do not show that a glaucoma model will work on new data. Normalization and color augmentation reduce the drop but do not remove it. Studies should report external validation and check the operating threshold, not only AUC.