واکنش در شیار خلفی راست برتر یک پاسخ مبتنی بر ویژگی ها به چهره زمانی را نشان می دهد

Abstract The face-selective region of the right posterior superior temporal sulcus (pSTS) plays an important role in analysing facial expressions. However, it is less clear how facial expressions are represented in this region. In this study, we used the face composite effect to explore whether the pSTS contains a holistic or feature-based representation of facial expression. Aligned and misaligned composite images were created from the top and bottom halves of faces posing different expressions. In Experiment 1, participants performed a behavioural matching task in which they judged whether the top half of two images was the same or different. The ability to discriminate the top half of the face was affected by changes in the bottom half of the face when the images were aligned, but not when they were misaligned. This shows a holistic behavioural response to expression. In Experiment 2, we used fMR-adaptation to ask whether the pSTS has a corresponding holistic neural representation of expression. Aligned or misaligned images were presented in blocks that involved repeating the same image or in which the top or bottom half of the images changed. Increased neural responses were found in the right pSTS regardless of whether the change occurred in the top or bottom of the image, showing that changes in expression were detected across all parts of the face. However, in contrast to the behavioural data, the pattern did not differ between aligned and misaligned stimuli. This suggests that the pSTS does not encode facial expressions holistically. In contrast to the pSTS, a holistic pattern of response to facial expression was found in the right inferior frontal gyrus (IFG). Together, these results suggest that pSTS reflects an early stage in the processing of facial expression in which facial features are represented independently.

. Introduction Interpreting the facial expressions of others is important to effective social interaction (Bruce & Young, 2012). Facial expressions result from characteristic patterns of movement of the facial muscles that can easily be seen in static photographs (usually showing the apex of the movement itself) or in videos (Johnston, Mayes, Hughes, & Young, 2013). However, little is known about how expressions are encoded at the neural level. The most widely-used neural model of face perception (Haxby, Hoffman, & Gobbini, 2000) proposes that the superior temporal sulcus (STS) is a key neural structure in the perceptual analysis of facial expressions, and this is borne out by a number of studies that have implicated STS in neural responses to expression (Calder and Young, 2005 and Psalta et al., 2014) and social perception from visual cues (Allison, Puce, & McCarthy, 2000). Relatively few studies address the question of how STS encodes expression. Said, Moore, Engell, Todorov, and Haxby (2010) were able to demonstrate that patterns of activation to different facial expressions across voxels in posterior STS (pSTS) were correlated with the rated perceptual similarities of the expressions themselves, suggesting that the functional organisation of pSTS reflects this underlying perceptual structure. Similarly, Harris, Young, and Andrews (2012) found that right pSTS responded to changes in facial expression regardless of whether or not these changes crossed or remained within emotional category boundaries, which again suggests a form of encoding that is largely driven by the perceptual input. Importantly, Harris, Young, and Andrews (2014) showed that right pSTS is relatively insensitive to contrast reversal, which implies that the critical perceptual input for pSTS involves feature shapes. Contrast reversal is known to have a dramatic effect on face identity recognition, but it has relatively little effect on the recognition of expression because information about feature shapes that is critical to interpreting facial expressions is conveyed through the position of edges that remain largely invariant to contrast reversal ( Bruce & Young, 1998). Here, we take the study of the perceptual representation used by pSTS a step further by asking whether it represents features such as the eyes and mouth independently from each other, or as part of a perceptual whole (the face). The critical test of holistic processing that we use for this purpose is the expression composite effect. Composite effects have been demonstrated in many studies of facial identity perception (Rossion, 2013 and Young et al., 1987), but their extension to understanding facial expression perception is less well-known. The paradigm involves combining the top half of one facial expression with the bottom half of another expression and determining whether this combination of different parts results in the perception of a new whole expression (Calder and Jansen, 2005, Calder et al., 2000, Palermo et al., 2011 and Prazak and Burgund, 2014). The critical test of holistic perception involves contrasting performance between images in which the top and bottom halves are aligned into a highly face-like overall configuration, or misaligned so that they are less face-like. Contrasting aligned and misaligned versions of composite images created from the top and bottom parts of different facial expressions makes it possible to differentiate responses based on face features, which will be equivalent across aligned and misaligned image variants, from holistic responses that will only be evident for aligned and not for misaligned images. In this study, we used the facial expression composite effect to investigate whether neural responses to facial expression in right pSTS reflect feature changes or are dependent on the face as a perceptual whole. To do this, we first established in a behavioural study that the stimuli and presentation parameters we intended to use in fMRI elicited a robust expression composite effect. We then compared neural responses in right pSTS to composite expressions in which the top (eye region) and bottom (mouth region) parts were aligned into an overall face-like configuration with neural responses to misaligned stimuli created by shifting one part horizontally with respect to the other (see Fig. 1). Misalignment still allows the separated parts of the face to be encoded as features, but it interferes with the integration of expressive information from the eye and mouth region into a perceptual whole (Calder et al., 2000). Examples of experimental stimuli used to create trial blocks in experiment 2. A) ... Fig. 1. Examples of experimental stimuli used to create trial blocks in experiment 2. A) Aligned conditions (top row: no change, middle row: bottom change, bottom row: top change); B) Misaligned conditions (top row: no change, middle row: bottom change, bottom row: top change). The stimuli used in experiment 1 involved sequentially presented pairs of images from each of the 6 types of trial block. Note that a small gap between the top and bottom halves of each stimulus emphasises where the parts are joined, even for the aligned images (cf. Rossion, 2013). Figure options Our fMRI experiment used a block design adaptation paradigm in which participants viewed blocks comprising a series of facial expressions that were all the same (no change condition) or that varied across the top half of each image (top change condition) or across the bottom half of each image (bottom change condition). During these blocks, participants were asked to fixate between the eyes (i.e., in the top half of each face) and further to encourage fixation they had to detect the presentation of an occasional small red spot at the fixation point. The no change condition, with identical stimuli throughout the block, served as a baseline that will lead to maximal adaptation of neural responses, and the top change or bottom change conditions measured any release from adaptation in neural regions that can encode these changes. The stimuli were aligned into overall face-like composites, or horizontally misaligned so that they were not face-like (see Fig. 1), allowing us to establish whether the pattern of neural responses across conditions involving no change, top change, or bottom change was dependent on the presence of a face-like (aligned) configuration.

. Results 3.1. Experiment 1 The aim of experiment 1 was to demonstrate the facial expression composite effect with the stimuli and presentation times to be used in the fMR-adaptation study. There were 6 conditions involving aligned or misaligned pairs with no change between the images, a bottom half change, or a top half change. Participants monitored the top half of pairs of face images to detect whether the facial expression in the top half remained the same, or was different across the two faces. First we measured the accuracy of responses when judging whether the top half of each image was the same or different. As participants were asked to make their judgements based on only the top half of each image, the correct responses in each condition were 'same' for no change pairs, 'same' for the bottom change pairs, and 'different' for the top change pairs. Percent correct responses were calculated for each condition for each participant, and then averaged across all participants to give an overall percent correct response measure. The data are displayed as percentage errors in Fig. 2A to facilitate comparison with reaction times shown in Fig. 2B. (A) Percent error responses for the same-different task in Experiment 1. The ... Fig. 2. (A) Percent error responses for the same-different task in Experiment 1. The critical result is reduced performance (increased errors) in the bottom change condition compared to the no change condition when the stimuli are aligned, but not when they are misaligned, even though in all 4 of these conditions the top halves of the stimuli are to be judged ‘same’ – the facial expression composite effect. The top change condition is less important because it involves a change in correct response (now 'different' instead of 'same'). (B) Median response times for the same-different task in experiment 1. RTs were longer for the bottom change condition compared to no change condition when the stimuli were aligned, but not misaligned – again demonstrating the expression composite effect. Figure options The proportion of correct responses was entered into a 2 × 3 repeated measures ANOVA with the factors Alignment (aligned, misaligned) and Condition (no change, top change, bottom change). The ANOVA showed a significant effect of Alignment [F(1,15) = 38.37, p < .001, partial eta squared = .72] and Condition [F(2,30) = 19.48, p < .001, partial eta squared = .57]. Bonferroni pairwise comparisons demonstrated that the effect of Alignment was driven by more accurate responses in the misaligned versus aligned conditions (p < .001). The effect of Condition was driven by more accurate responses in the no change versus top change (p < .001) and bottom change (p = .001) conditions. However, these main effects were both qualified by the presence of a significant Alignment × Condition interaction [F(2,30) = 10.82, p < .001, partial eta squared = .42]. Paired t-tests demonstrated this was a result of lower accuracy in the bottom change condition when the stimuli were aligned, compared to misaligned [t(15) = −5.54, p < .001] but no difference between the no change aligned and misaligned conditions [t(15) = .432, p = .672]. This part of the interaction is the critical test of the facial composite effect, because in all four of these conditions participants were making equivalent responses (that the top halves were the 'same'). In addition, there was also a non-significant trend demonstrating lower accuracy for the top change condition when the stimuli were aligned, compared to misaligned [t(15) = −1.86, p = .083]. Whilst of interest, this is less crucial because the correct response has now switched to 'different'. We also measured response times to each condition. Median RTs were taken for each condition, for each participant and an overall median RT was calculated for each condition across all participants (Fig. 2B). These median RTs were entered into a 2 × 3 repeated measures ANOVA with the factors Alignment (aligned, misaligned) and Condition (no change, top change, bottom change). This ANOVA demonstrated significant main effects of Alignment [F(1,15) = 18.24, p = .001, partial eta squared = .55] and Condition [F(2,30) = 16.36, p < .001, partial eta squared = .52]. Bonferroni pairwise comparisons demonstrated the effect of Alignment was driven by longer RTs when the stimuli were aligned, compared to misaligned (p = .001) and the effect of Condition was driven by a longer RT in both top change (p ≤ .001) and bottom change (p < .001) conditions relative to no change. Again, interpretation of these main effects needs to be qualified by a significant Alignment × Condition interaction [F(2,30) = 11.62, p < .001, partial eta squared = .44]. Paired t-tests demonstrated this was due to longer response times in the aligned versions of both top change and bottom change conditions when compared to their misaligned counterparts [bottom change: t(15) = 4.69, p < .001, top change: t(15) = 3.04, p < .001]. No difference was seen in the response times between the aligned and misaligned versions of the no change condition [t(15) = −1.54, p = .145]. Parallelling the analysis of accuracy data, the slower response times in the aligned compared to misaligned version of the bottom change condition, and the lack of difference in response time for the no change condition, illustrate the key components of the face composite effect. In sum, behavioural results from the RT and accuracy data show the facial expression composite effect where participants find it more difficult to judge the top half of the images as the same when the bottom half is changing and the two halves of each image are aligned into an overall facial configuration, compared to when they are in a misaligned form. 3.2. Experiment 2 The aim of this experiment was to investigate properties of the right pSTS response to facial expressions, using conditions comparable to those in the behavioural experiment 1. The principal focus of the analysis was pSTS because of its hypothesised role in facial expression perception in the leading neural model of face perception, (Haxby et al., 2000), and on right rather than left pSTS because right pSTS is more reliably identified at the individual participant level with our functional localiser scan and has therefore been targeted in previous studies (Harris et al., 2012 and Harris et al., 2014). To parallel experiment 1, there were 6 different types of block in the experimental scan, involving aligned or misaligned pairs with no change between the images, a bottom half change, or a top half change. In order to check whether participants were watching the top halves of the stimuli throughout the experiment, as instructed, they were given the task of pressing a response button every time they saw a small red dot presented at the fixation point. Performance on this red dot detection task was high, with a mean accuracy of 99% correct responses and mean RT of 447 msec. To confirm that there were no differences in overall attentional demands between aligned and misaligned stimuli, the average response times to aligned and misaligned conditions for each participant were entered into a paired t-test. There was no significant difference in response times to the red dot, t(21) = 1.39, p = .18. The pSTS, FFA and OFA were localised in the left and right hemispheres using the independent functional localiser scan. The OFA and FFA could be identified in both the left and right hemispheres for 23/26 participants. In contrast to the OFA and FFA, the pSTS could be reliably identified in the right hemisphere of 22/26 participants, but in the left hemisphere for only 15/26 participants. This relatively poor face responsiveness of left pSTS may be due to its possible role in more audiovisual integration of vocal and facial speech signals (Calvert, 2001, Pelphrey et al., 2005 and Wright et al., 2003). Average MNI coordinates and number of voxels for each localised ROI are provided in Table 1. Table 1. Average MNI coordinates in mm (mean and SE), size in voxels, and number of participants where the region could be identified, for each ROI. ROI Coordinate No. of voxels No. of participants x y z Right OFA 41 ± 1 −80 ± 2 −15 ± 1 187 26 Left OFA −41 ± 1 −83 ± 1 −14 ± 1 107 23 Right FFA 41 ± 1 −56 ± 1 −23 ± 1 223 26 Left FFA −40 ± 1 −60 ± 2 −23 ± 1 114 23 Right pSTS 51 ± 1 −61 ± 2 1 ± 1 110 23 Table options There was no effect of hemisphere for the OFA [F(1,22) = .16, p = .696] or FFA [F(1,22) = 1.58, p = .221], so the data from the left and right hemispheres of these regions were combined. For pSTS, we used only the region localised in the right hemisphere. In terms of Haxby et al.'s (2000) neural model of face perception, results for the pSTS and FFA are the most instructive, as these lie on separate neural pathways considered to be critically involved in the perception of expression (pSTS) or to be involved in other aspects of face perception (FFA). Data for the pSTS and FFA are therefore summarised in Fig. 3. The OFA was considered as of less interest because it lies on both neural pathways in Haxby et al.'s (2000) model, but data from the OFA were analysed, for completeness. Overall mean MR time series for each condition for aligned and misaligned ... Fig. 3. Overall mean MR time series for each condition for aligned and misaligned stimuli, and peak % BOLD signal change for right pSTS (row A), FFA (row B) and OFA (row C). Analysis of the responses in right pSTS revealed a smaller peak response in the no change condition compared to both the bottom change (p = .001) and top change conditions (p = .008), with no difference between the bottom and top change conditions. This pattern held for aligned and misaligned stimuli. In FFA, there was only a main effect of Alignment, with a higher peak response to aligned than misaligned stimuli (p = .021). Error bars represent standard error of the mean. Figure options First, we took the time series data for each participant and averaged these across participants to give an overall mean time series for each condition, for each ROI (Fig. 3). We then looked at the peak responses in the right pSTS, which form the study's principal focus of interest (Fig. 3, panel A). A 2 × 3 ANOVA with the factors Alignment (aligned, misaligned) and Condition (no change, bottom change, top change) demonstrated a significant effect of Condition [F(2,44) = 7.62, p = .001], but not of Alignment [F(1,22) < 1]. The Alignment × Condition interaction was not significant [F(2,44) < 1]. The effect of Condition was driven by a smaller peak percentage signal change in the no change condition compared to both the bottom change [t(22) = −3.75, p = .001] and top change conditions [t(22) = −2.93, p = .008], with no difference between the signal change in the bottom and top change conditions [t(22) = .301, p = .797]. This pattern is consistent with a feature-based response, with no evidence of the critical interaction between Alignment and Condition that would demonstrate holistic perception. It is important to note that in this study, we looked at the response across all facial expressions. Although our design does not allow for the data to be explored in this way, it would be interesting to look at the response for each individual expression. This would be particularly interesting as some facial expressions are more recognisable from their bottom halves, and some from their top halves (Calder et al., 2000). The FFA showed a different pattern of results to the pSTS (Fig. 3, panel B). A 2 × 3 ANOVA showed a significant effect of Alignment [F(1,25) = 6.11, p = .021], but only a borderline effect of Condition [F(2,50) = 2.56, p = .088]. The Alignment × Condition interaction was not significant [F(2,50) < 1]. The effect of Alignment was driven by a significantly higher peak percent signal change to the aligned compared to misaligned stimuli [t(25) = 2.47, p = .021]. The OFA did not produce any findings that reached conventional levels of statistical significance (Fig. 3). There was no effect of Alignment [F(1,25) < 1], and after Greenhouse-Geisser correction for a violation of sphericity [χ2(2) = 9.03, p = .011] only a borderline effect of Condition [F(1.523,38.07) = 3.32, p = .059]. There was no Alignment × Condition interaction [F(2,50) < 1]. To determine if other regions showed a holistic response, we also conducted a whole brain analysis. The % error and response time data from Experiment 1 were used as regressors to identify regions that might show a holistic response. The resulting group statistical parametric map identified 2 clusters of activity, in the right inferior frontal gyrus (IFG) and in the right fusiform gyrus. Table 2 shows the peak voxel intensity, co-ordinates and size of the ROIs based on the % error and RT data. Table 2. Peak intensity and MNI coordinates (mm) for maximally active voxel, and size in voxels for each ROI identified using the mean RT and % error data from experiment 1 as a regressor. ROI Peak intensity (z score) Coordinate No. of voxels x y z % Error Right Fusiform 4.86 38 −50 −22 771 Right IFG 3.90 48 4 18 411 RT Right Fusiform 4.97 40 −50 −24 656 Right IFG 4.09 48 6 18 654 Table options These data were used to create masks of the regions identified (right fusiform, and right IFG). We took the time series data for each participant and averaged across participants to give an overall mean time series for each condition, for each ROI. The peak responses for each condition for each ROI were then calculated. As can be seen from Table 2, the peak intensities were very similar for both the ROIs identified using the RT and % error data. This was also reflected in the peak response to each individual condition, therefore we have only presented the % error regressor data for illustration purposes, in Fig. 4. The right IFG shows the classic pattern demonstrated in the expression composite effect – a higher response to bottom change when the face is aligned, compared to when misaligned. It also shows a smaller response to the no change compared to the change conditions. In contrast, the fusiform gyrus shows a more general overall difference in responsiveness between aligned and misaligned images. This is consistent with the known involvement of fusiform cortex in the holistic perception of faces (Andrews et al., 2010 and Kanwisher et al., 1997), but does not imply holistic processing of expression per se. Overall mean peak % BOLD signal change for each condition for aligned and ... Fig. 4. Overall mean peak % BOLD signal change for each condition for aligned and misaligned stimuli for the right IFG (A), and right fusiform (B). Regions defined using the % error data from experiment 1 as a regressor. Error bars represent standard error of the mean.