Exploring crossmodal perceptual enhancement and integration in a sequence-reproducing task with cognitive priming

Abstract

Leveraging the perceptual phenomenon of crossmoal correspondence has been shown to facilitate peoples information processing and improves sensorimotor performance. However for goal-oriented interactive tasks, the question of how to enhance the perception of specific Crossmodal information, and how Crossmodal information integration takes place during interaction is still unclear. The present paper reports two experiments investigating these questions. In the first experiment, a cognitive priming technique was introduced as a way to enhance the perception of two Crossmodal stimuli, in two conditions respectively, and their effect on sensory-motor performance was observed. Based on the results, the second experiment combined the two Crossmodal stimuli in the same interfaces in a way that their correspondence congruency was mutually exclusive. The same priming techniques was applied as a manipulating factor to observe the Crossmodal integration process. Results showed that first, the Crossmodal integration during interaction can be enhanced by the priming technique, but the effect varies according to the combination of Crossmodal stimuli and the types of priming material. Moreover, peoples subjective evaluations towards priming types were in contradiction with their objective behavioural data. Second, when two Crossmodal sequences can be perceived simultaneously, results suggested different perceptual weights are possessed by different participants, and the perceptual enhancement effect was observed only on the dominant one, the pitch-elevation. Furthermore, the Crossmodal integration tended to be integrated in a selective manner without priming. These results contribute design implications for multisensory feedback and mindless computing.