دانلود مقاله ISI انگلیسی شماره 29676
ترجمه فارسی عنوان مقاله

تصاویر بصری دقیق: نقش نگاه خیره در تصویرسازی ذهنی و حافظه

عنوان انگلیسی
Scrutinizing visual images: The role of gaze in mental imagery and memory
کد مقاله سال انتشار تعداد صفحات مقاله انگلیسی
29676 2014 11 صفحه PDF

Publisher : Elsevier - Science Direct (الزویر - ساینس دایرکت)

Journal : Cognition, Volume 131, Issue 2, May 2014, Pages 263–283

ترجمه کلمات کلیدی
تصویرسازی - حافظه - چشم ردیابی - تصویب -
کلمات کلیدی انگلیسی
Imagery, Memory, Eye-tracking, Enactment,
پیش نمایش مقاله
پیش نمایش مقاله  تصاویر بصری دقیق: نقش نگاه خیره در تصویرسازی ذهنی و حافظه

چکیده انگلیسی

Gaze was monitored by use of an infrared remote eye-tracker during perception and imagery of geometric forms and figures of animals. Based on the idea that gaze prioritizes locations where features with high information content are visible, we hypothesized that eye fixations should focus on regions that contain one or more local features that are relevant for object recognition. Most importantly, we predicted that when observers looked at an empty screen and at the same time generated a detailed visual image of what they had previously seen, their gaze would probabilistically dwell within regions corresponding to the original positions of salient features or parts. Correlation analyses showed positive relations between gaze’s dwell time within locations visited during perception and those in which gaze dwelled during the imagery generation task. Moreover, the more faithful an observer’s gaze enactment, the more accurate was the observer’s memory, in a separate test, of the dimension or size in which the forms had been perceived. In another experiment, observers saw a series of pictures of animals and were requested to memorize them. They were then asked later, in a recall phase, to answer a question about a property of one of the encoded forms; it was found that, when retrieving from long-term memory a previously seen picture, gaze returned to the location of the part probed by the question. In another experimental condition, the observers were asked to maintain fixation away from the original location of the shape while thinking about the answer, so as to interfere with the gaze enactment process; such a manipulation resulted in measurable costs in the quality of memory. We conclude that the generation of mental images relies upon a process of enactment of gaze that can be beneficial to visual memory.

مقدمه انگلیسی

In his book Inquiries into Human Faculty and its Development (1883), Sir Francis Galton discussed mental imagery as a special ability of human visual memory. Specifically, he wondered whether mental images could be “so clear and sharp as […] to be scrutinized with nearly as much ease and prolonged attention as if they were real objects.” Galton prompted his informants to “think of some definite object—suppose it is your breakfast-table as you sat down to it this morning—and consider carefully the picture that rises before your mind’s eye […] Is the image dim or fairly clear? […] Are all the objects pretty well defined at the same time, or is the place of sharpest definition at any one moment more contracted than it is in a real scene?” Reports about the “definition” of the imagined breakfast items varied very much across individuals; however, a common report was that one or two objects would appear much more distinct than the others but these could come out clearly if attention be paid to them. Thus, different objects were not clear all at once but only successively, by focusing attention on them at different time points. About a century later, although accounts of imagery did not rely any longer exclusively on introspective reports, the modern cognitive psychologists also concluded that whenever we generate a visual image of an object, the different parts of the object are not clear all at once but only successively (e.g., Hebb, 1968 and Neisser, 1976). Kosslyn, 1980 and Kosslyn, 1994 has also put forward an influential computational model for visual imagery, according to which each part of an image is added in successive steps (Kosslyn et al., 1988 and Kosslyn et al., 1983). Visual images take time both to generate and to inspect and, in many respects, they strongly resemble the normal perception of objects at close range, where a high-resolution perceptual representation of the object cannot be achieved in a single glance but a series of eye movements must bring into ‘foveal’ focus the different parts of the object. One remarkable finding of several studies of imagery is that while imagining something there appears to be a lot of motor activity, which resembles the exploratory movements typically made during perceptual scrutiny of an object or scene. Jacobson (1932; see also Totten, 1935) had originally observed with a galvanometer that engaging in imagery (e.g., recollection) resulted in the measurement of action potentials in muscle groups that were specific to the body part which was imaginatively moved (e.g., during visual imagination, movements of the eye-balls was registered, while when thinking, one could register brief contractions in muscles of tongue). Moreover, several researchers have noticed a remarkable similarity in the duration of imagined actions compared to the time it takes to perform them (e.g., Decety, 1996, Decety et al., 1989, Jeannerod, 1994 and Parsons, 1987). These findings clearly implicate the presence of motor processing during imagery, although the motor processes would often seem to constitute only a subset of those activated during overt movement (Ellis, 1995). According to recent studies, gaze patterns (i.e., fixations and/or direction of saccades) that are measured in real time during recollection of a previous event look remarkably similar to the scanpaths during a perceptual recognition test of the same scene, despite the fact that when thinking about the episode there is nothing at all to look at on a blank computer screen. This phenomenon has been repeatedly observed in a variety of studies (e.g., Moore, 1903: Altmann, 2004, Brandt and Stark, 1997, Brandt et al., 1989, de’Sperati, 2003, Gbadamosi and Zangemeister, 2001, Hollingworth, 2005, Humphrey and Underwood, 2008, Jeannerod and Mouret, 1962, Johansson et al., 2006, Laeng and Teodorescu, 2002, Laeng et al., 2007, Martarelli and Mast, 2013, Renkewitz and Jahn, 2012 and Spivey and Geng, 2001). It would seem that, when retrieving a visual image or episode, not only there occur spontaneous eye movements but these tend to reflect the content of the original scene. Deckert (1964) had observed that participants instructed to imagine a beating pendulum developed pursuit ocular movements of a frequency comparable to the frequency of a previously seen real pendulum. Intriguingly, studies of rapid eye movements or REM during sleep also would seem to show some relationship between the types of eye movements and the content of dreams (e.g., Aserinsky and Kleitman, 1953, Dement and Kleitman, 1957, Doricchi et al., 2007, Hong et al., 1997, Hong et al., 2009 and Roffwarg et al., 1962) as well as time-locked activity within primary visual cortex (Miyauchi, Misaki, Kan, Fukunaga, & Koike, 2009). At a first glance, the above phenomena are puzzling because it seems a meaningless expenditure of bodily energy and cognitive effort to move about the eyes when there is nothing to be seen. Purposeful saccades that cannot garner any visual input appear completely paradoxical in relation to normal visual processing, since the pattern of saccadic movements during perception seems to be purposefully guided towards visual information or ‘objects’ that are relevant for the cognitive system at that particular time (e.g., Einhäuser et al., 2008, Findlay and Gilchrist, 2003, Hayhoe and Ballard, 2005, Noton and Stark, 1971a, Noton and Stark, 1971b, Rothkopf et al., 2007, Rucci et al., 2007, Schütz et al., 2012, Stark and Ellis, 1981, Trommershäuser et al., 2009 and Yarbus, 1967). Importantly, eye movements indicate the occurrence of shifts in spatial attention (Craighero et al., 2004, Deubel and Schneider, 1996, Henderson, 1992, Moore and Fallah, 2001, Rolfs et al., 2011 and Shepherd et al., 1986) and covert visual attention may consist in the motor preparation of an eye movement (Rizzolatti et al., 1983 and Rizzolatti et al., 1987). Hence, oculomotor activity could overload the cognitive system and/or interfere with other processes (cf. Loftus, 1972). Since the early days of research on mental imagery, both Francis Galton and Alfred Binet (Hadamard, 1945, pp. 72–73) had suggested that there may be an antagonism between the vividness or detail of a visual image and the presence of other activities. A solution to the above puzzle is to assume that, contrary to the idea that such “empty” looks during recollection and imagination are either deleterious or irrelevant to cognition, they may actually serve some useful function. There is growing evidence for shared mechanisms of perception and imagery (e.g., Kan et al., 2003 and Kosslyn and Thompson, 2000). In addition, the idea that perception is “active” or “embodied” has been gaining strength over the years within the cognitive sciences and neurosciences (Barsalou, 1999, Ellis, 1995, Findlay and Gilchrist, 2003, Gibbs, 2006, Gibson, 1979, Pezzulo et al., 2001 and Pulvermüller and Fadiga, 2010). This perspective stresses the idea that the visual system does not merely register its environment but explores it and poses questions by “grasping” objects with the eyes and/or hands (Ballard et al., 1997, Castelhano et al., 2009, Karn and Hayhoe, 2000 and Land et al., 1999). If perception and imagery share processing mechanisms, then also imagery may be “active” in the sense that adjustments of the body organs, even in a vacuum, could play a significant role in the retrieval of internally stored information. A straightforward hypothesis, already well-formulated by Hebb (1968), is that such an empty gaze serves the function of assisting the mental re-construction of a representation. According to Hebb (1968, p. 470), “if the image is a reinstatement of the perceptual process it should include the eye movements […] and if we can assume that the motor activity, implicit or overt, plays an active part we have an explanation of the way in which the part-images are integrated sequentially”. Neisser (1976) also speculated that the act of constructing an image would require eye movements like those originally made in perceiving, because imagery is a process of visual synthesis and construction, much like perception. The fundamental “Hebbian” idea behind the present study is that eye fixations can provide a sort of “scaffolding structure” for generating a visual image part-by-part. As put by Mast and Kosslyn (2002), eye movements could play an important role in allowing one to visualize a montage, a composite created on the basis of memories of multiple fixations. In other words, a single object’s image may be constructed in a manner that is not that different from imagining a scene; since an object has a categorical spatial structure between its parts (Laeng, Shah, & Kosslyn, 1999), these can be treated as separate units or “objects”. Thus, gaze could trigger sequences of memories and could also help to position correctly each image of a part relative to other parts. For example, we may have vivid imagery of, say, a cat, when we go through (some of) the motions of looking at something and determining that it is a cat, even though there is actually no cat (Thomas, 2011). Thus, contrary to the idea that motoric activity during imagery may be an epiphenomenon, a meaningless spill-over of mental activity while thinking to be back in a previously encountered situation, which in itself could bear no meaningful effect on cognitive processing (e.g., Marks, 1973 and Teichner et al., 1978), we believe that the present phenomena actually reflect something very important about the nature of mental representations. Most current models of episodic memory do posit that one of the key functions of imagery is to allow reconstructing the past and, in particular, to generate specific predictions based on past experience (Addis et al., 2007, Hassabis et al., 2007, Moulton and Kosslyn, 2009, Schacter et al., 2007 and Schacter et al., 2008). That is, imagery allows making explicit and accessible aspects of a specific situation. If someone’s gaze is engaged during recollection, despite being actually “looking at nothing”, this might actually tell us a great deal about mechanisms involved in memory recall (Ferreira et al., 2008 and Ryan et al., 2000). Specifically, memory representations are based on integrating input from various sources with spatial information, which would seem to be registered by default in working memory as part of a dynamic motor system (Altmann, 2004, Altmann and John, 1999, Ballard et al., 1997, Hodgson et al., 2002, Logie, 1995, Richardson et al., 2009 and Richardson and Kirkham, 2004). Thus, the visual system automatically registers a spatial index or pointer to a position in the visual field as a core element of an episodic trace, also in circumstances in which actions are not required, the location information is not relevant for solving the task, and there is no intention or demand to learn the spatial information (Laeng et al., 2007 and Richardson and Spivey, 2000). Kent and Lamberts (2008) have proposed that memory retrieval is generally elicited by “mental simulation” (Barsalou, 1999); supposedly, when the integrated memory episode is reactivated at a later time, the spatial index relating to an object or part will also be automatically retrieved (Bourlon et al., 2011 and Hoover and Richardson, 2008), which in turn triggers the eyes to move to the indexed location in which the part originally appeared. As Ballard et al. (1997, p. 724) point out: “Because humans can fixate on an environmental point, their visual system can directly sample portions of three dimensional space […] and as a consequence, the brain’s internal representations are implicitly referred to an external point.” Thus, gaze direction may indicate a retrieval attempt for a specific item of information (Renkewitz & Jahn, 2012). In fact, outside of on-screen laboratory experiments, locations in the environment are rarely completely empty. Therefore, gaze might garner useful contextual visual cues (like noticing an empty chair) when attempting to recall visual information. Finally, returning the eyes to the former location of an object could also improve memory for information associated with that object (e.g., Hollingworth, 2006 and Johansson and Johansson, 2013), especially if spatial information contributes to maintaining the continuity and integrity of the “object file” or event (Hommel, 2004, Hommel et al., 2001 and Kahneman et al., 1992). In support of the above idea that the motoric activity during imagery plays a functional role, there exists some evidence that the accuracy of memory retrieval can be disrupted when someone who is holding an image in mind is restrained from making an eye movement or deliberately moves in an image-irrelevant way (e.g., Andrade et al., 1997, Antrobus et al., 1964, Barrowcliff et al., 2004, Gunter and Bodner, 2008, Postle et al., 2006, Ruggieri, 1999 and Singer and Antrobus, 1965). Several studies have shown the same phenomenon with other movement types; e.g., the recall of an imagined path can be disrupted by a concurrent movement of the arm (Quinn, 1994). A counterclockwise manual rotation hinders the concomitant clockwise mental rotation of a visual object and vice versa; however, a counterclockwise mental rotation of a visual object does facilitate a clockwise mental rotation (Wexler, Kosslyn, & Berthoz, 1998). Demarais and Cohen (1998) observed that, while solving transitive inference problems with the terms left/right or above/below, participants spontaneously made more horizontal than vertical saccades during the former task but they showed the reverse pattern for the latter. Glenberg and Kaschak (2002) found that, when judging whether a sentence was sensible (e.g., “close the drawer”), participants had difficulty making such a judgment if required to make a response in the opposite direction. Dijkstra, Kaschak, and Zwaan (2008) have found that participants could retrieve more efficiently autobiographical information when their body positions while being queried were similar to the body position they had during the original event. In eye-tracking studies, when participants perform a problem solving tasks and simultaneously their eye movements are “guided” either according to a scanpath related to the problem’s solution or in an irrelevant way, the former gaze patterns lead to successful problem-solving than the latter ones (e.g., Grant and Spivey, 2003 and Thomas and Lleras, 2007). Laeng and Teodorescu (2002) specifically found that memory suffered when spontaneous fixations during recall were prevented by enforcing fixation on a central cross at the time the participant attempted to answer a question regarding a previously seen object, which strongly suggests that the eye movements occurring during image generation are not epiphenomenal or a consequence of the experiment’s task demands (Jolicoeur & Kosslyn, 1985). Instead, they strongly suggest that, by disrupting a spontaneous action pattern, the memory system may be hindered in the retrieval of the details of a mental representation and that they play a functional role in the process of recollecting and re-constructing a previous perception. Consistently with the findings of Laeng and Teodorescu’s (2002; Experiment 2), successive studies have found evidence, by forcing fixation during retrieval, that eye movements played a functional role for memory, since this procedure reduced episodic memory performance (Johansson et al., 2012, Johansson and Johansson, 2013 and Mäntylä and Holm, 2006).