|
Multisensory (auditory-visual) perceptions: speech and non-speech
analysis systems| old_uid | 1463 |
|---|
| title | Multisensory (auditory-visual) perceptions: speech and non-speech
analysis systems |
|---|
| start_date | 2006/06/23 |
|---|
| schedule | 14h30-16h |
|---|
| online | no |
|---|
| location_info | salle des Actes |
|---|
| summary | Audition and vision are classically studied as separate perceptual systems. Recently, however, a multi-disciplinary body of work suggests that auditory and visual systems can no longer be considered as independent analytical systems. Here, I will present further evidence for this claim in both speech and non-speech contexts.
In the first part of the talk, I will focus on audio-visual (AV) temporal adaptation, where after exposure to desynchronized AV stimuli (e.g. an auditory stimulus precedes a visual stimulus by a few tens of milliseconds), participants’ subjective simultaneity of the AV events shifts towards the adaptor (e.g. audio). Magnetoencephalography (MEG) was used to functionally investigate hypothesized cortical plasticity underlying this effect. The temporal variability in AV events elicited rapid habituation of the auditory-evoked magnetic fields. The reconstruction of the underlying cortical sources in time and frequency space revealed a large-scale multisensory network that includes
sensory-specific and multisensory cortices as well as frontal areas.
This network showed local synchronization states in classic (<60Hz) and in high gamma frequency ranges (>70Hz) as a function of participants perceiving, recalling or imagining the AV events. These results suggest that AV perceptual adaptation is mediated by a large-scale adaptative network.
I will then focus on AV speech perception in the context of the ‘analysis-by-synthesis’ framework. Classic information-theoretic models of speech perception do not presently predict McGurk type effects (e.g. an audio [pa] dubbed onto a face articulating [ka] is perceived as [ta]). Psychophysical and electroencephalographic (EEG) data will be
described supporting the view that AV speech presents specific computational challenges. For instance, an EEG study will be reported suggesting that the natural dynamics of AV speech and the degree of saliency in visual speech contextualize early auditory analysis.
Additionally, psychophysical results suggest that the temporal resolution of AV speech integration results from a parallel analysis of speech inputs on two-different time scales.
In short, the notion of perceptual inference in multisensory perception will be put forward, and distinctions between AV perceptions will be outlined in terms of levels of representation and computation. |
|---|
| responsibles | Pressnitzer |
|---|
| |
|