Learning to See the Elephant in the Room: Self-Supervised Context Reasoning in Humans and AI

Abstract

Humans rarely perceive objects in isolation but interpret scenes through relationships among co-occurring elements. How such contextual knowledge is acquired without explicit supervision remains unclear. Here we combine human psychophysics experiments with computational modelling to study the emergence of contextual reasoning. Participants were exposed to novel objects embedded in naturalistic scenes that followed predefined contextual rules capturing global context, local context and crowding. After viewing short training videos, participants completed a "lift-the-flap" task in which a hidden object had to be inferred from the surrounding context under variations in size, resolution and spatial arrangement. Humans rapidly learned these contextual associations without labels or feedback and generalised robustly across contextual changes. We then introduce SeCo (Self-supervised learning for Context Reasoning), a biologically inspired model that learns contextual relationships from complex scenes. SeCo encodes targets and context with separate vision encoders and stores latent contextual priors in a learnable external memory module. Given contextual cues, the model retrieves likely object representations to infer hidden targets. SeCo outperforms state-of-the-art self-supervised learning approaches and predicts object placements most consistent with human behaviour, highlighting the central role of contextual associations in scene understanding.