واکنش N170 به حالات چهره توسط تناسب عاطفی بین ابراز هیجانی و تصویر عاطفی قبلی تعدیل شده است

Abstract Does contextual affective information influence the processing of facial expressions already at the relatively early stages of face processing? We measured event-related brain potentials to happy and sad facial expressions primed by preceding pictures with affectively positive and negative scenes. The face-sensitive N170 response amplitudes showed a clear affective priming effect: N170 amplitudes to happy faces were larger when presented after positive vs. negative primes, whereas the N170 amplitudes to sad faces were larger when presented after negative vs. positive primes. Priming effects were also observed on later brain responses. The results support an early integration in processing of contextual and facial affective information. The results also provide neurophysiological support for theories suggesting that behavioral affective priming effects are based, at least in part, on facilitation of encoding of incoming affective information.

Introduction Facial expressions are of utmost importance for social interaction as they convey information about other individuals’ emotions and social intentions (Fridlund, 1991, Keltner et al., 2003 and Russell et al., 2003). In the laboratory, perception of facial expressions is typically investigated by presenting isolated pictures of faces to the participants. However, in every-day life, an individual's face is usually not perceived in isolation. Instead, it appears in rich context of information emanating from other people and the physical environment surrounding the expressor as well as multichannel information from the expressor herself. This influence can be significant for the perception of facial expressions, especially when the context information contains affective meaning. Indeed, the expressor's own emotional expressions in the vocal prosody (de Gelder and Vroomen, 2000, Hietanen et al., 2004, Massaro and Egan, 1996 and Vroomen et al., 2001), body posture (Meeren et al., 2005), and hand movements (Hietanen and Leppänen, 2008) modulate the recognition of his/her facial expressions. Also, surrounding people's facial expressions (Masuda et al., 2008), emotion provoking stories (Carroll and Russell, 1996), film clips (Niedenthal et al., 2000), visual affective scenes (Aviezer et al., 2008 and Righart and de Gelder, 2008b), and odors (Leppänen and Hietanen, 2003) influence the facial expression recognition. A typical finding in these studies is that facial expression recognition is faster and more accurate when the context is affectively congruent rather than incongruent with the facial expression (affective congruency effect). The contextual effects are observed even when the participants are explicitly asked to attend on the facial expressions and ignore the context information. These findings give evidence for the automatic nature of processing of affective information from all available sources and integrating it with the face processing. From cognitive neuroscience viewpoint an interesting issue relates to the stage of processing where the contextual information starts to interact with the processing of facial expressions. Is it an early (automatic) or a late (more controlled) process? Because of its excellent temporal resolution, measuring of event-related potentials (ERPs) is a potentially useful method to study this issue. Visual ERP studies have identified a specific ERP response which has been proposed to be particularly sensitive to faces (Bentin et al., 1996, Itier and Taylor, 2004 and Rossion and Jacques, 2008). This component, known as N170, is recorded over occipitotemporal regions peaking between 140 and 200 ms after stimulus onset, and characterized by a bilateral temporal negative deflection. The N170 response is considered to reflect the early visual processing of structural, configurally represented information in the face (Bentin et al., 1996, Eimer, 2000 and Itier and Taylor, 2004). Importantly for the present study, the N170 response (or its magnetoencephalographic counterpart, M170) may also be sensitive to facial emotional expressions (Batty and Taylor, 2003, Blau et al., 2007, Caharel et al., 2005, Eger et al., 2003, Japee et al., 2009, Leppänen et al., 2007, Williams et al., 2006, Vlamings et al., 2009 and Wronka and Walentowska, 2011), although there are some previous studies lacking to show this effect (e.g., Eimer and Holmes, 2002, Eimer et al., 2003 and Holmes et al., 2003). However, these differences could be related to attentional effects, e.g., whether the faces were presented to the attentional focus, and whether the task specifically required processing of facial expressions. In all the studies where faces were centrally presented and the participants were instructed to direct their attention to facial expressions, modulation of N170 amplitude by facial expressions has been observed ( Caharel et al., 2005, Japee et al., 2009, Leppänen et al., 2007 and Wronka and Walentowska, 2011). There are a couple of previous studies which have investigated whether the integration of facial expressions and other types of affective information is reflected in the early visual ERP responses. Meeren et al. (2005) showed face–body compound stimuli in which the face and body components were expressing fear or anger, either congruently or incongruently. The P1 response (peaking on average at 115 ms) was sensitive to the affective congruency between the body and face stimuli: the P1 was larger for incongruent vs. congruent face–body compound stimuli. However, the authors themselves suggested that the enhanced P1 responses to incongruent compound stimuli could be related to the deviation from the normal face–body composition in the incongruent cases, and that this anomalous conflict between facial and bodily expression could have attracted more attention than emotion-congruent stimuli prompting more elaborate analysis in early visual areas. Moreover, it is debatable whether the P1 response reflects processing of a high-level percept of a face at all, but rather some low-level features (Rossion and Caharel, 2011 and Rossion and Jacques, 2008). No effect of affective congruency between the face and body components was found for the amplitude of the N170 in the study by Meeren and colleagues. Righart and de Gelder (2006) presented fearful and neutral faces overlaid on pictures of fearful and neutral natural scenes. The results gave evidence for the context effect on N170 responses: N170 was larger in the context of fearful than neutral scenes. However, although this enhancement was larger for fearful than neutral expressions, it was observed for both fearful and neutral faces. Thus, this result can be considered to provide only partial support for a congruency effect. In a following study, Righart and de Gelder (2008a) presented again face–context compound stimuli to the participants. This time the scenes were fearful, happy or neutral, the faces were fearful or happy. Again, the electrophysiological results replicated the previous ones: the N170 response was larger for both expressions in the context of fearful vs. other scenes, although this enhancement seemed to be larger for fearful than happy faces. There was no effect of affective context on the P1 responses in either of these studies. In sum, despite the strong behavioral evidence of emotional context information influencing the recognition of facial expressions, previous studies have not been able to show a clear affective congruency effect on the face-sensitive N170 response. In the above-mentioned visual ERP studies, the stimulus faces were presented simultaneously with and embedded in the affective context information. In the present study, we decided to use another well-known paradigm for investigating the affective context effect: affective priming. In affective priming, an affective target is presented after (e.g., 300 ms) an affective prime. The results typically show that affectively congruent prime–target pairs result in shorter response times to targets (e.g., positive/negative evaluative decisions) than affectively incongruent prime–target pairs (for reviews, see Fazio, 2001 and Klauer and Musch, 2003). There are a few previous studies which have measured ERPs to facial expressions in affective priming paradigms. However, in these studies the focus has been on investigating the affective congruency effects on longer latency ERPs, especially on N400 and LPP (late positive potential) responses, which are known to show congruency effects also in classic semantic priming studies (Besson et al., 1992). And indeed, these studies have shown enhanced N400 and LPP responses to emotionally incongruent prime–target pairs (Krombholz et al., 2007, Paulmann and Pell, 2010 and Werheid et al., 2005). Krombholz et al. (2007) also investigated affective congruency effect specifically on N170 responses to facial expressions (primed by emotion words “anger” and “happiness”) but no effect was found. Paulmann and Pell (2010) primed facial expressions by emotional prosody vocalizations. No effect of affective congruency on N170 responses was found in this study either (see Fig. 4, in Paulmann and Pell, 2010). In Werheid et al. study, the target faces (primed by another expressive face) did not evoke a typical N170 response at all. To sum up, we investigated whether the early face-sensitive N170 response is modulated by the affective congruency between the emotional expression displayed on the face (target) and the affective content of a preceding natural scene (prime). The primes were pictures with complex, emotional scenes evaluated as emotionally positive or negative. The target stimuli were facial expressions of happiness and sadness. The prime–target stimulus-onset-asynchrony (SOA) was 300 ms. The presentation of the prime and target stimuli was combined in such a way that half of the trials were affectively congruent and another half emotionally incongruent. The participants’ task was to categorize the faces as happy or sad. We measured both behavioral (reaction time and recognition accuracy) and ERP responses. Recognition accuracies were expected to be higher and response times shorter for facial expressions preceded by affectively congruent vs. incongruent primes. More importantly, we expected the N170 responses to be larger in amplitude for facial expressions preceded by affectively congruent vs. incongruent primes. The hypothesis for N170 response enhancement for affectively congruent stimuli was based on previous findings from studies of affective context effects on N170 amplitudes (e.g., Righart and de Gelder, 2006 and Righart and de Gelder, 2008a) and on findings from studies showing that, for example, as a function of the facial expression intensity, the N170 amplitudes ( Sprengelmeyer and Jentzsch, 2006) as well as the recognition accuracy (e.g., Hoffmann et al., 2010) are enhanced.

. Results 3.1. Behavioral results The mean recognition accuracies and mean response times to happy and sad facial expressions as a function of prime category are presented in Fig. 1. For the mean recognition accuracies, the analysis showed that both main effects were significant. The recognition accuracy was slightly higher after the negative (91.1%) than positive (88.8%) primes, F(1,19) = 4.8, p < .041, η2 = .201, and, overall, the recognition accuracy was higher for happy (97.1%) than sad (82.9%) faces, F(1,19) = 36.1, p < .001, η2 = .655. Most importantly, the interaction between the main effects was significant, F(1,19) = 18.9, p < .001, η2 = .499. Further analyses showed that the priming effect was significant both for the recognition of happy and sad expressions. The happy faces were recognized more accurately after the positive (98.4%) than negative (95.8%) primes, t(19) = 3.17, p < .005, whereas the sad expressions were recognized more accurately after the negative (86.6%) than positive (79.3%), t(19) = 3.59, p < .002. The priming effect on recognition accuracy (%congruent − %incongruent) was significantly stronger for sad (7.3%) than happy faces (2.7%), t(19) = 2.2, p < .041. Mean (and SEM) recognition accuracy (a) and response time (b) for the happy and ... Fig. 1. Mean (and SEM) recognition accuracy (a) and response time (b) for the happy and sad facial target expressions primed by the positive and negative scenes (**p < .01; ***p < .001). Figure options For the mean response times to happy and sad facial expressions, the analysis showed that both main effects were significant. Overall, the reaction times were shorter after positive (705.3 ms) than negative primes (726.9 ms), F(1,19) = 14.2, p < .001, η2 = .427. The responses were also faster to happy (649.6 ms) than sad target faces (782.6 ms), F(1,19) = 67.2, p < .001, η2 = .780. Again, the interaction between the main effects was significant, F(1,19) = 25.5, p < .001, η2 = .573. The happy faces were recognized faster after the positive (617.8 ms) than negative (681.4 ms) primes, t(19) = 6.6, p < .001, whereas the sad expressions were recognized faster after the negative (772.3 ms) than positive (792.8 ms) primes although the difference was only marginally significant for the sad faces, t(19) = 1.9, p < .068. The priming effect (RTinconguent − RTcongruent) was significantly stronger for happy (63.6 ms) than sad faces (20.5 ms), t(19) = 3.8, p < .001. 3.2. Electrophysiological results 3.2.1. N170 amplitude Overall, the N170 amplitude was larger for happy (−4.2 μV) than sad (−3.9 μV) faces, F(1,19) = 4.9, p < .04, η2 = .206. The N170 amplitudes were also larger over the right (−4.8 μV in P8) than left (−3.4 μV in P7) hemisphere, F(1,19) = 7.3, p < .02, η2 = .277. Importantly, the interaction between the effects of prime and target was significant, F(1, 19) = 29.9, p < .001, η2 = .611. None of the other effects were significant (all ps > .2). Further analyses showed that the priming effect was significant both for the happy and sad expressions. The N170 amplitudes for happy faces were larger after the positive (−4.4 μV) than negative (−4.1 μV) primes, t(19) = 2.37, p < .03, whereas the N170 amplitude for sad faces was larger after the negative (−4.2 μV) than positive (−3.7 μV) primes, t(19) = 3.82, p < .001. There was no difference in the N170 amplitude priming effect for the happy and sad faces (p > .28). The interaction between the prime and target was observed also when analyzed for the amplitude difference between the N170 and the preceding P1 component (see Fig. 2, Fig. 3 and Fig. 4). ERP waveforms to happy faces primed by positive and negative scenes. The ... Fig. 2. ERP waveforms to happy faces primed by positive and negative scenes. The investigated ERP components are indicated by arrows. Time = 0 ms indicates the target onset. The voltage distribution map shows the difference between mean amplitude values to positively and negatively primed faces within the time-window of the N170 response. Figure options ERP waveforms to sad faces primed by positive and negative scenes. The ... Fig. 3. ERP waveforms to sad faces primed by positive and negative scenes. The investigated ERP components are indicated by arrows. Time = 0 ms indicates the target onset. The voltage distribution map shows the difference between mean amplitude values to negatively and positively primed faces within the time-window of the N170 response. Figure options The affective priming effect on the N170 responses to happy (left) and sad ... Fig. 4. The affective priming effect on the N170 responses to happy (left) and sad (right) faces primed by positive (in blue) and negative (in red) scenes. The waveforms represent averaged data across recording channels P7 and P8. Time = 0 ms indicates the target onset. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.) Figure options 3.2.2. N170 latency The main effect of target was significant. Overall, the N170 latencies were longer for sad (181.6 ms) than happy (173.0 ms) faces, F(1,19) = 12.3, p < .002, η2 = .393. The main effect of prime, F(1,19) = 3.4, p < .080, η2 = .152, and the interaction between the effects of prime and target approached significance, F(1,19) = 3.51, p < .076, η2 = .156. None of the other effects were significant (all ps > .2). The latency of the N170 response tended to be shorter after the positive (175.5 ms) than negative (179.1 ms) primes. Because of the marginally significant interaction between the prime and target, the priming effect was analyzed separately for the happy and sad faces. These analyses showed that, for the happy faces, the N170 latency was significantly shorter after the positive (169.5 ms) than negative (176.5 ms) primes, t(19) = 2.6, p < .019. For the sad faces, there was no effect of prime category on the N170 latency (p > .8). It should be noted that when the analysis was run for the difference in the latencies between N170 and P1, there was no interaction between the prime and target. 3.2.3. P1 response The analysis on the P1 amplitudes showed that the main effects of prime and laterality were significant. Overall, the P1 amplitude was larger after negative (1.2 μV) than positive (0.9 μV) primes, F(1,19) = 8.2, p < .01, η2 = .302. The P1 amplitudes were also larger over the right (1.6 μV in P8) than left (0.5 μV in P7) hemisphere, F(1,19) = 14.2, p < .001, η2 = .429. These main effects were qualified by a significant three-way interaction between the main effects, F(1,19) = 4.8, p < .05, η2 = .203. For electrode P7, a prime × target ANOVA showed a marginal interaction, F(1,19) = 3.6, p < .08, η2 = .161. Pairwise comparisons showed that the P1 amplitudes for happy faces were marginally larger after the negative (0.6 μV) than positive (0.3 μV) primes, t(19) = 1.94, p < .07. There was no effect of prime on the P1 amplitudes to sad faces (p > .8). For electrode P8, there was only a main effect of prime, F(1,19) = 8.3, p < .08, η2 = .304. Overall, the P1 amplitude was larger after negative (1.9 μV) than positive (1.4 μV) primes (see Fig. 2, Fig. 3 and Fig. 4). The analysis on the P1 latency data showed a marginal interaction between prime and target, F(1,19) = 4.3, p < .06, η2 = .184. Pairwise comparisons showed that, for the happy faces, the P1 latency was marginally shorter after the positive (108.2 ms) than negative (112.8 ms) primes, t(19) = 2.0, p < .07. For the sad faces, there was no effect of prime category on the P1 latency (p > .4). 3.2.4. Early posterior negativity (EPN) The main effects of prime and target were significant. Overall, EPN was shifted in the negative direction after negative (−0.97 μV) vs. positive (−0.62 μV) primes, F(1,19) = 20.0, p < .001, η2 = .513. The amplitudes were also more negative to sad (−0.90 μV) than happy (−0.69 μV) faces, F(1,19) = 5.6, p < .03, η2 = .228. The interaction between prime and target was significant, F(1,19) = 38.2, p < .001, η2 = .668. Follow-up analyses showed that, for the happy faces, EPN amplitude was more negative after the negative (−1.10 μV) than positive (−0.28 μV) primes, t(19) = 9.5, p < .001. For the sad faces, there was no effect of prime category on EPN amplitude (p > .3) (see Fig. 2 and Fig. 3). 3.2.5. N400 The analysis showed that all the main effects were significant. Overall, N400 was more positive after negative (0.93 μV) than positive (0.60 μV) primes, F(1,19) = 15.5, p < .001, η2 = .450. The N400 waveform was also more positive for sad (0.89 μV) than happy (0.61 μV) faces, F(1,19) = 7.6, p < .02, η2 = .285. The N400 amplitude was more positive for electrode Fz (1.22 μV) than Cz (0.28 μV), F(1,19) = 38.1, p < .001, η2 = .667. The interaction between prime and target was significant, F(1,19) = 16.1, p < .001, η2 = .459. Because the 3-way interaction between prime, target, and electrode site was also significant, F(1,19) = 5.3, p < .04, η2 = .217, we decided to analyze the priming effect separately for the electrodes Fz and Cz. For electrode Fz, both main effects were significant (prime: p < .01; target: p < .001). The interaction between prime and target was highly significant, F(1,19) = 20.9, p < .001, η2 = .523. For the happy faces, the N400 response was shifted in the negative direction for congruent (0.55 μV) vs. incongruent (1.43 μV) trials, t(19) = 6.5, p < .001. For the sad faces, the response did not differ after congruent (1.43 μV) and incongruent (1.46 μV) primes (p > .8). For electrode Cz, the main effect of prime was significant (p < .01) and, more importantly, the interaction between prime and target was also significant, F(1,19) = 6.5, p < .02, η2 = .255. For the happy faces, the N400 response was shifted in the negative direction for the congruent (−0.42 μV) vs. incongruent (0.50 μV) trials, t(19) = 4.70, p < .001. For the sad faces, the response did not differ after congruent (0.35 μV) and incongruent (0.31 μV) primes (p > 7) (see Fig. 2 and Fig. 3). 3.2.6. LPP The analysis showed that the main effect of target was significant. Overall, the amplitude of this response was shifted in the positive direction after sad (−0.4 μV) vs. happy (−0.7 μV) faces, F(1,19) = 4.7, p < .05, η2 = .199. The interaction between prime and target was also significant, F(1,19) = 14.4, p < .001, η2 = .431. Because of this interaction the priming effect was analyzed separately for happy and sad faces. For the sad faces, the amplitude was shifted in the positive direction after incongruent (−0.3 μV) vs. congruent (−0.6 μV) primes, t(19) = 2.79, p < .012. For the happy faces, there was no difference in the amplitude after incongruent (−0.6 μV) and congruent (−0.7 μV) primes (p > .2) (see Fig. 2 and Fig. 3).