Combined perception of emotion in pictures and musical sounds

Evaluation of emotional scenes requires integration of information from different modality channels, most frequently from audition and vision. Neither the psychological nor neural basis of auditory–visual interactions during the processing of affect is well understood. In this study, possible intera...

Full description

Saved in:
Bibliographic Details
Published in:Brain research 2006-01, Vol.1070 (1), p.160-170
Main Authors: Spreckelmeyer, Katja N., Kutas, Marta, Urbach, Thomas P., Altenmüller, Eckart, Münte, Thomas F.
Format: Article
Language:English
Subjects:
Adult
Affective processing
Attention
Auditory Perception
Biological and medical sciences
Communication
Crossmodal integration
Emotion
Emotions
Event-related potential
Evoked Potentials
Female
Fundamental and applied biological sciences. Psychology
Humans
Male
Mental Processes
Music - psychology
Somesthesis and somesthetic pathways (proprioception, exteroception, nociception)
interoception
electrolocation. Sensory receptors
Vertebrates: nervous system and sense organs
Visual Perception
Voice
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Evaluation of emotional scenes requires integration of information from different modality channels, most frequently from audition and vision. Neither the psychological nor neural basis of auditory–visual interactions during the processing of affect is well understood. In this study, possible interactions in affective processing were investigated via event-related potential (ERP) recordings during simultaneous presentation of affective pictures (from IAPS) and affectively sung notes that either matched or mismatched each other in valence. To examine the role of attention in multisensory affect-integration ERPs were recorded in two different rating tasks (voice affect rating, picture affect rating) as participants evaluated the affect communicated in one of the modalities, while that in the other modality was ignored. Both the behavioral and ERP data revealed some, although non-identical, patterns of cross-modal influences; modulation of the ERP-component P2 suggested a relatively early integration of affective information in the attended picture condition, though only for happy picture–voice pairs. In addition, congruent pairing of sad pictures and sad voice stimuli affected the late positive potential (LPP). Responses in the voice affect rating task were overall more likely to be modulated by the concomitant picture's affective valence than vice versa.
ISSN:0006-8993
1872-6240
DOI:10.1016/j.brainres.2005.11.075