پردازش ادراکی، طبقه، و عاطفی لبخند مبهم حالات صورت

Abstract Why is a face with a smile but non-happy eyes likely to be interpreted as happy? We used blended expressions in which a smiling mouth was incongruent with the eyes (e.g., angry eyes), as well as genuine expressions with congruent eyes and mouth (e.g., both happy or angry). Tasks involved detection of a smiling mouth (perceptual), categorization of the expression (semantic), and valence evaluation (affective). The face stimulus display duration and stimulus onset asynchrony (SOA) were varied to assess the time course of each process. Results indicated that (a) a smiling mouth was visually more salient than the eyes both in truly happy and blended expressions; (b) a smile led viewers to categorize blended expressions as happy similarly for upright and inverted faces; (c) truly happy, but not blended, expressions primed the affective evaluation of probe scenes 550 ms following face onset; (d) both truly happy and blended expressions primed the detection of a smile in a probe scene by 170 ms post-stimulus; and (e) smile detection and expression categorization had similar processing thresholds and preceded affective evaluation. We conclude that the saliency of single physical features such as the mouth shape makes the smile quickly accessible to the visual system, which initially speeds up expression categorization regardless of congruence with the eyes. Only when the eye expression is later configurally integrated with the mouth, will affective discrimination begin. The present research provides support for serial models of facial expression processing

1. Introduction In a categorical approach to facial affect, emotional expressions are conceptualized as discrete entities that can be subsumed under six basic categories: fear, anger, sadness, happiness, disgust, and surprise (Ekman, 1994). Although this conceptualization is not devoid of its own limitations (see critical reviews in Barrett (2006) and Barrett, Gendron, and Huang (2009)), it has been widely adopted by prior research on the recognition of facial expressions, and studies have generally used prototypical examples of the six categories as stimuli. In real life, however, there is an enormous idiosyncrasy and variability across individuals and social contexts, where ambiguous expressions are frequently encountered (Carroll and Russell, 1997 and Scherer and Ellgring, 2007). For example, Ekman (2001) identified at least 18 different types of smiles, and proposed that there may be as many as 50 in all. However, the evidence regarding the perception, categorization, and affective processing of non-prototypical expressions has remained elusive. In the present study we examined whether the processing of ambiguous expressions with a smile, but non-happy eyes, involves the same mechanisms and stages as that of the prototypical expressions. To investigate the recognition of ambiguous expressions, studies have used genuine blends (Nummenmaa, 1988), hybrids (Schyns & Oliva, 1999), morphed (Calder, Rowland, et al., 2000), and composite (Calder, Young, Keane, & Dean, 2000) face stimuli. In composite faces, the top half of a face conveying one expression is fused with the bottom half of another expression. The resulting facial configuration is thus a blend of two expressions and therefore becomes ambiguous. In the current study we employed this approach by aligning the bottom half of a happy face with the top half of non-happy faces (either angry, sad, fearful, disgusted, surprised, or neutral) of the same individual. This produced blended expressions with a smiling mouth but non-happy eyes. For comparison, we also used intact faces conveying prototypical happy or non-happy expressions, in which the eye region was congruent with the mouth region, as both belonged to the same category (and individual), and therefore these expressions were genuine and unambiguous. With this approach, we explored (a) the extent to which the presence of a smile—even though incongruent with other facial components—can bias the recognition of an expression as happy, (b) whether featural or configural processing is involved in the categorization and affective evaluation of blended expressions, and (c) how such bias develops over time for the extraction of perceptual, categorical, and emotional information. 1.1. Role of a smile in facial expression processing From a theoretical standpoint, smiles provide a useful and well-established model for studying the processing of blended facial expressions. First, happy faces are recognized more accurately and faster than all the other basic expressions (Calder, Young, et al., 2000, Calvo and Lundqvist, 2008, Calvo and Nummenmaa, 2009, Juth et al., 2005, Leppänen and Hietanen, 2004, Loughead et al., 2008, Milders et al., 2008, Palermo and Coltheart, 2004 and Tottenham et al., 2009). Facial happiness is also the most consistently identified expression across different cultures (Russell, 1994). Second, the smile is a critical feature supporting the recognition advantage of happy faces. Whereas a smiling mouth is a necessary and sufficient criterion for categorizing faces as happy, the eye region makes only a modest contribution (Calder, Young, et al., 2000, Kontsevich and Tyler, 2004, Leppänen and Hietanen, 2007, Nusseck et al., 2008 and Smith et al., 2005). Third, the importance of the smile in facilitating expression recognition has been attributed to its high visual saliency and diagnostic value (Calvo and Nummenmaa, 2009 and Calvo et al., 2010). Because the smiling mouth is a salient or conspicuous feature, the smile attracts the first eye fixation more likely than any other region of the six basic expressions (Calvo & Nummenmaa, 2008). In addition, because of its distinctiveness or diagnostic value, the smile is systematically associated with facial happiness and is absent in all other expression categories (Calvo & Marrero, 2009). Such a single diagnostic feature can thus be used as a shortcut for a quick categorization of a face as happy. By assuming the contribution of the saliency and distinctiveness of a smile, we addressed two questions. First, is the smiling mouth so salient and distinctive that it overrides the processing of other facial components, such as the eye region, even when these are inconsistent in meaning with the smile? If so, the presence of a smiling mouth would bias viewers towards judging blended facial expressions as happy, regardless of other expressive sources. This issue has obvious theoretical and practical importance, as in many everyday situations people smile without necessarily being happy. The smile is a complex, multifunctional signal, which can also reflect mere politeness or even conceal negative motives (embarrassment, dominance, etc.), depending on the combination with other facial signals (see Ambadar et al., 2009 and Niedenthal et al., 2010). It is, nevertheless, possible that, because of its saliency and distinctiveness, a smiling mouth “dazzles” the viewers and prevents them from noticing less salient yet informative facial cues such as frowns, which would be necessary to interpret the smile accurately and react with adaptive behavior in time. Second, does the categorization of an ambiguous, as well as a genuine, smiling face involve only perceptual processing—either single feature detection or configural pattern recognition—of a salient mouth, or also extraction of positive affect? Furthermore, can affect be obtained from feature analysis or does it require configural analysis, and what is the relative time course of these processes? There is considerable evidence that faces (Richler, Mack, Gauthier, & Palmieri, 2009) and facial expressions (Calder, Young, et al., 2000) are processed configurally or holistically, that is, coded as unitary objects, in an integrated representation that combines the different face parts. Studies have, nevertheless, also shown that the category of an emotional expression can be inferred from the separate analysis of single distinctive facial components (Ellison and Massaro, 1997 and Fiorentini and Viviani, 2009). Probably, both views are complementary, with some expressions being more dependent on holistic and others relying more on analytic processing (Tanaka, Kaiser, Butler, & Le Grand, in press). In any case, for both the configural and the featural conceptualization, it is possible that expression recognition is performed solely on the basis of some perceptual pattern or a single visual cue from the face image, without retrieving any affective meaning. It remains unresolved whether emotional representations are also activated during perceptual processing and used for expression categorization, and how this applies to blended facial expressions. 1.2. The current study To investigate whether a smiling mouth can overshadow other facial regions and override their processing even when these are incongruent with the smile, we used composite faces (Calder, Young, et al., 2000, Leppänen and Hietanen, 2007 and Tanaka et al., in press), in which non-happy (e.g., angry) eyes were combined with a smiling mouth, thus conveying blended expressions. These faces were then compared with intact faces conveying prototypical expressions (happy, angry, etc.) in several tasks. In Experiment 1, we used a categorization task in which participants responded whether each face looked happy or not. To determine the role of configural and featural processing, the face stimuli were presented upright or spatially inverted. In Experiments 2 and 3, we aimed to distinguish between affective and perceptual processes. To this end, we used priming tasks in which happy, neutral, or blended face primes were followed by an emotional probe scene. Participants responded whether the scene was pleasant or unpleasant (affective priming task) or whether there was any person smiling in the probe scene (perceptual priming task). Depending on the emotional congruence and the visual similarity between the prime and the probe, priming effects (i.e., faster responding to the probe following the happy or the blended prime, relative to the neutral prime) allowed us to tease apart affective and perceptual processing of the genuine and the blended expressions. Finally, in Experiment 4, we directly compared the performance on all three tasks (perceptual, categorization, and affective) within the same experimental design. A major aim of this study was to investigate the relative time course of perceptual, categorical, and affective processing of blended and prototypical expressions. To examine when discrimination between genuinely happy and blended expressions with a smile occurs, in Experiments 2 and 3, we varied the stimulus onset asynchrony (SOA; 170, 340, 550, and 800 ms) between a prime face and a probe scene. The lower boundary of 170 ms was motivated by the earliest event-related potential (ERP; N170 component) that reflects facial structural encoding (i.e., differentiation between faces and non-face objects), about which there is a debate on whether it is also sensitive to emotional expression (i.e., differentiation of emotional from non-emotional faces; see Eimer and Holmes, 2007 and Schacht and Sommer, 2009). Discrimination among emotional expressions is likely to start later, as reflected in the N300 and P300 (from 250 to 500 ms) ERP components (Luo, Feng, He, Wang, & Luo, 2010). Similarly, conscious recognition of basic expressions can be accomplished in ∼300 ms (Calvo and Nummenmaa, 2009 and Calvo and Nummenmaa, 2011), which thus represents a critical time point for comparison with blended expressions. Finally, affective retrieval is predicted to occur later than categorization. In fact, affective priming by happy or liked faces has been found between 300 and 750 ms from face onset (Calvo et al., 2010, Lipp et al., 2009 and Nummenmaa et al., 2008). Our 550- and 800-ms SOA conditions will thus explore a possible delay for blended expressions. In a complementary approach to estimate the processing time course, in Experiment 4, we varied the display duration of the face stimuli (20, 40, 70, and 100 ms), while participants judged whether or not (a) the mouth was smiling (perceptual), (b) the face was happy (categorization), and (c) the expression was pleasant (affective).

.