خواندن موسیقی خاموش : تصویرسازی شنوایی و انتقال روش ویژو تونال در خوانندگان و غیر خوانندگان

In daily life, responses are often facilitated by anticipatory imagery of expected targets which are announced by associated stimuli from different sensory modalities. Silent music reading represents an intriguing case of visuotonal modality transfer in working memory as it induces highly defined auditory imagery on the basis of presented visuospatial information (i.e. musical notes). Using functional MRI and a delayed sequence matching-to-sample paradigm, we compared brain activations during retention intervals (10 s) of visual (VV) or tonal (TT) unimodal maintenance versus visuospatial-to-tonal modality transfer (VT) tasks. Visual or tonal sequences were comprised of six elements, white squares or tones, which were low, middle, or high regarding vertical screen position or pitch, respectively (presentation duration: 1.5 s). For the cross-modal condition (VT, session 3), the visuospatial elements from condition VV (session 1) were re-defined as low, middle or high “notes” indicating low, middle or high tones from condition TT (session 2), respectively, and subjects had to match tonal sequences (probe) to previously presented note sequences. Tasks alternately had low or high cognitive load. To evaluate possible effects of music reading expertise, 15 singers and 15 non-musicians were included. Scanner task performance was excellent in both groups. Despite identity of applied visuospatial stimuli, visuotonal modality transfer versus visual maintenance (VT > VV) induced “inhibition” of visual brain areas and activation of primary and higher auditory brain areas which exceeded auditory activation elicited by tonal stimulation (VT > TT). This transfer-related visual-to-auditory activation shift occurred in both groups but was more pronounced in experts. Frontoparietal areas were activated by higher cognitive load but not by modality transfer. The auditory brain showed a potential to anticipate expected auditory target stimuli on the basis of non-auditory information and sensory brain activation rather mirrored expectation than stimulation. Silent music reading probably relies on these basic neurocognitive mechanisms.

While inner mental life hitherto appeared essentially private and inaccessible to empirical research, subjective reports of inner experience can nowadays be objectively cross-checked to some degree by functional neuroimaging on the basis of correlated brain activation patterns (Kosslyn et al., 2001, McNorgan, 2012, Zatorre, 2003 and Zatorre and Halpern, 2005). Notational audiation, i.e. the inner experience of music in the absence of acoustic input as induced by silent music reading, provides one of the most intriguing examples of mental imagery (Godøy & Jørgensen, 2001) and appears well suited for neurocognitive research as the score precisely defines the music to be imagined (e.g. rhythm, pitch, harmonies, sound, and emotional quality). Meanwhile, a broad stream of neurocognitive findings have corroborated the notion of subjective auditory experience during music imagery by showing concomitant activation of auditory brain areas in the absence of acoustic input (Brodsky et al., 2008, Gordon, 1975, Halpern, 2001, Hubbard, 2010, Zatorre, 2003 and Zatorre and Halpern, 2005). From a working memory perspective, silent music reading translates into a case of visuospatial-to-tonal modality transfer in working memory (Calvert, 2001 and Evans and Treisman, 2010). Thereby, modality transfer is supposed to be a latent cognitive mechanism which facilitates later responses to target stimuli which were previously announced by associated stimuli from a different modality (Calvert, 2001). Visuo-tonal modality transfer provides an opportunity to test contradictory implications of the two most influential working memory models, the multi-component model ( Baddeley and Hitch, 1974 and Repovš and Baddeley, 2006) and the embedded-processes model ( Cowan, 1999 and Cowan et al., 2008). Of note, tonal material can largely be treated like other phonological material in this context ( Akiva-Kabiri et al., 2009, Schulze et al., 2011 and Williamson et al., 2010). While both accounts appear consistent with a visual-to-auditory brain activation shift during silent music reading, they make different predictions regarding motor processes. As the multi-component model stresses the modality specificity of the storage units it essentially requires (subvocal) articulatory processes to transfer visually presented material into a phonological code before it may enter the phonological loop ( Muller and Knight, 2006 and Repovš and Baddeley, 2006). In contrast, the embedded-processes model predicts a shift of the focus of attention from visual to auditory aspects of given information, but no motor processes. Thus, the activation or non-activation of speech-related motor brain areas (e.g. supplementary-motor area, SMA) during visuotonal modality transfer becomes a critical marker for the validity of both accounts. In this functional magnetic resonance imaging (fMRI) study, we assessed neural mechanisms behind auditory imagery as induced by silent music reading, i.e. visuotonal modality transfer. We predicted that visuotonal modality transfer is associated with a shift of activation from visual to auditory brain areas during the retention interval of a delayed matching-to-sample task. More precisely, we tested the following hypotheses: (1) Visual stimuli rapidly gain a potential to activate auditory brain areas if they are associated with tonal information by re-defining them as notes indicating pitch. (2) Activation of auditory brain areas by visually presented notes in a cross-modal task might be even stronger than activation of auditory cortices elicited by tones in a unimodal maintenance task due to the additional requirement of self-generating a tonal representation during modality transfer (Suchan, Linnewerth, Koster, Daum, & Schmid, 2006). (3) In the absence of task-relevant acoustic input, visual stimulation might not only activate secondary and higher but even the primary auditory cortex (McNorgan, 2012). (4) The proposed visual-to-auditory activation shift also implies “deactivation” or inhibition of visual brain areas during increased auditory activation ( Azulay et al., 2009, Hairston et al., 2008 and Laurienti et al., 2002), consistent with the notion of highly automated visual processing but almost complete absorption in musical imagery during score reading in experts. (5) Activation of motor brain areas during visual-phonological modality transfer might indicate executive processes as predicted by the multi-component model of working memory. (6) We included singers with high levels of sight-singing skills and unskilled lay persons to test the hypothesis that expertise in terms of extended practice and higher performance in visuotonal modality transfer affects the proposed visual-to-auditory brain activation shift during silent music reading. Overall, we hypothesized that imagery might indicate an anticipatory role of sensory brain areas in response facilitation, beyond mere reception and storage of given information.

Mental imagery has been characterized as “preparedness to perceive” (Neisser, 1976) or as “perception in a hypothetical manner” (Ryle, 1949). In our study, auditory imagery was induced when subjects transferred visually presented information into a tonal representation to facilitate later response to expected target stimuli. The cross-modal transfer was mirrored by a shift of activation from visual to auditory brain areas while no evidence for support from extrasensory brain areas was obtained. The activation-shift pattern was found in the total sample and only appeared to be enhanced in experts. Thus, sensory brain areas generally appear to be capable of early multisensory integration and sensory activation might rather reflect representations of expected target stimuli than the actual sensory input. We assume that music imagery utilizes these fundamental neurocognitive mechanisms.