Tricking Synthetic Image Detectors with Amortized Generative Attacks

Abstract

Rapid advances in image synthesis have increased the need for reliable synthetic-image detectors, yet their robustness to transferable attacks remains incomplete and understudied. Existing evaluations generally examine whether an attack constructed against one detector transfers to another, but rarely isolate how its effectiveness changes when the attacked images originate from different generators. We conduct a systematic evaluation of three representative attacks: two iterative perturbation-based methods, MIG and MuMoDIG, and the amortized generative attack Tricker. Across four detectors and five generators spanning diverse architectures, we introduce a two-axis evaluation protocol that independently varies the surrogate detector used to construct or train the attack and the generator supplying its input images. We assess attack effectiveness, cross-detector transferability, perceptual quality, runtime, and interaction with common post-processing operations. We revisit Tricker and show that its data and training requirements can be substantially reduced, enabling broader and more efficient evaluation. Our results demonstrate that transferability is not a detector-only property: the input-generator distribution strongly influences both evasion effectiveness and attacked-image quality. MIG and MuMoDIG exhibit pronounced dependence on the detector--generator pair, whereas Tricker provides more uniform white-box evasion and image quality. We additionally observe that post-processing maintains the asymmetric transferability topology. These findings expose a neglected dimension of attack evaluation and motivate transfer-aware protocols that jointly account for detector and generator variation. We will release our code, evaluation data, and trained attack models with a responsible-use statement.