مجموعه NimStim حالات صورت: قضاوت از شرکت کنندگان در تحقیق آموزش ندیده

Abstract A set of face stimuli called the NimStim Set of Facial Expressions is described. The goal in creating this set was to provide facial expressions that untrained individuals, characteristic of research participants, would recognize. This set is large in number, multiracial, and available to the scientific community online. The results of psychometric evaluations of these stimuli are presented. The results lend empirical support for the validity and reliability of this set of facial expressions as determined by accurate identification of expressions and high intra-participant agreement across two testing sessions, respectively.

Introduction The literature on human social and emotional behavior is rich with studies of face processing. An online search in databases like PubMed for “face perception” or “face processing” to date results in over 7000 relevant papers. A similar search for “facial expression” results in over 5000 hits. Integral to these studies is the availability of valid and reliable face stimuli. This paper describes a set of face stimuli (the NimStim Set of Facial Expressions — available to the scientific community at http://www.macbrain.org/resources.htm) and reports validity and reliability for this set based on ratings from healthy, adult participants. Facial expressions and their interpretation are a topic of interest to researchers because of the links between emotional experience and facial expression. It has been argued that expressions of facial emotion have communicative value (Darwin, 1872) with phylogenic roots (lzard, 1971). Both the production (Ekman and Friesen, 1978) and the interpretation of facial expressions (Schlosberg, 1954 and Russell and Bullock, 1985) have been examined empirically. Previous research using existing sets (Ekman and Friesen, 1976) has established much of what is known about face processing. However, the parameters of existing sets may not always satisfy the objectives of the experiment (Erwin et al., 1992). For example, the number of stimuli may be too few (Winston et al., 2002), the stimulus set may not have enough racial or ethnic diversity, or there might not be an appropriate comparison or baseline expression (Phillips et al., 1998). Issues like these have motivated researchers to create their own stimuli (Hart et al., 2000, Phelps et al., 2000, Gur et al., 2002, Batty and Taylor, 2003, Phelps et al., 2003 and Tanaka et al., 2004). The goal of creating the NimStim Set of Facial Expressions was to provide a set of facial expressions that address these issues. A number of features of the set are advantageous for researchers who study face expression processing. Perhaps the most important is the racial diversity of the actors. Studies often show that the race or ethnicity of a model impacts face processing both behaviorally (Elfenbein and Ambady, 2002 and Herrmann et al., 2007) and in terms of the underlying neurobiology of face processing (Hart et al., 2000, Phelps et al., 2000, Golby et al., 2001 and Lieberman et al., 2005). This modulation by race or ethnicity is not identified for all populations (Beaupre and Hess, 2005) and may be driven by experience (Elfenbein and Ambady, 2003) and bias (Phelps et al., 2000 and Hugenberg and Bodenhausen, 2003). To address such findings, face sets have been developed that consist of models from various backgrounds. For example, the JACFEE set (Ekman and Matsumoto, 1993–2004) consists of Japanese and Caucasian models, Mandal's (1987) set consists of Indian models, Mandal et al.'s (2001) set consists of Japanese models, the Montreal Set of Facial Displays of Emotion (Beaupre et al., 2000) consists of French Canadian, Chinese, and sub-Saharan African models, and Wang and Markham's (1999) set consists of Chinese models. Unlike these other sets, the NimStim set provides one uniform set of Asian-American, African-American, European-American, and Latino-American actors, all photographed under identical conditions. Additionally, because these actors all live in the same metropolitan city within the U.S., the subtle differences that accompany expressions when posed by individuals from different countries are minimized (Ekman and Friesen, 1969 and Matsumoto et al., 2005). The NimStim set has attributes, described below, that are not typically found in other sets that include models from non-European populations. There are four distinguishing attributes of the set that build on previously established sets. First, the NimStim set is available in color, and it contains a large number of stimuli and a large variety of facial expressions. Whereas most sets contain roughly 100 or fewer stimuli (Mandal, 1987, Ekman and Matsumoto, 1993–2004, Wang and Markham, 1999 and Mandal et al., 2001), the NimStim set contains 672, consisting of 43 professional actors, each modeling 16 different facial poses, including different examples of happy, sad, disgusted, fearful, angry, surprised, neutral, and calm. Secondly, a neutral expression is included in this set. The neutral expression is sometimes included in other facial expression sets (Ekman and Friesen, 1976, Erwin et al., 1992 and Ekman and Matsumoto, 1993–2004), but is often omitted, particularly in sets that include models from different racial and ethnic backgrounds (Mandal, 1987, Wang and Markham, 1999, Beaupre et al., 2000 and Mandal et al., 2001). The inclusion of the neutral expression is important since neutral is often a comparison condition, particularly in neuroimaging studies (Breiter et al., 1996 and Thomas et al., 2001). Thirdly, the NimStim set contains open- and closed-mouth versions of each expression, which can be useful to experimentally control for perceptual differences (e.g., toothiness) that can vary from one expression to another, as this featural difference may bias responses (Kestenbaum and Nelson, 1990). Having closed- and open-mouth versions also facilitates morphing between various expressions by reducing blending artifacts. Lastly, since positively valenced faces are generally lower in arousal than negatively valenced faces and can present an arousal/valence confound, this set includes three degrees of happy faces (e.g., closed-mouth, open-mouth, and high arousal/exuberant). A final distinguishing feature of this set is the inclusion of a calm face. Studies examining the perception of facial expressions often use neutral faces as the comparison face (Breiter et al., 1996 and Vuilleumier et al., 2001). However, there is evidence to suggest that neutral faces may not always be perceived as emotionally neutral (Donegan et al., 2003, Somerville et al., 2004 and Iidaka et al., 2005), especially for children (Thomas et al., 2001 and Lobaugh et al., 2006). Researchers have artificially generated other comparison faces (i.e., 25% happy) to address this concern (Phillips et al., 1997). Within the NimStim Set, a calm expression category is provided, which is perceptually similar to neutral, but may be perceived as having a less negative valence. Here we provide data validating the use of the calm face. Validation of the entire set was accomplished by asking participants to label each stimulus. A different method of rating face stimuli involves having highly trained raters use facial action units to make judgments about the expressions (Ekman and Friesen, 1977 and Ekman and Friesen, 1978). This method is very useful for establishing the uniformity of expression exemplars. The merit of the method used in this article is that the ratings were obtained from untrained volunteers, who are characteristic of those in face expression processing studies. In other words, the approach taken in this study to establish whether a certain expression was perceived as the intended expression was to measure concordance between the subjects' labels and the intended expressions posed by the actors (the validity measure) as well as the intra-participant test–retest reliability. We hypothesized that the participants' judgments of these stimuli would provide empirical support for the reliability and validity of this new set of facial expressions.

3. Results 3.1. Validity There were two validity measures (proportion correct and kappa scores) for each stimulus, thus resulting in 672 proportion correct and kappa scores. These 672 proportion correct and kappa scores are presented individually in Supplemental Tables 2a and 2b and in aggregate by actor (43 actors × 2 mouth positions = 86 scores) in Supplemental Table 3, but by expression (17 scores) for succinctness within this manuscript (Table 1; closed and open mouth separately). The overall proportion correct was high (mean = 0.81 (S.D. = 0.19), median = 0.88). The overall concordance between raters' labels and the intended expressions was also high (mean kappa across stimuli = 0.79 (S.D. = 0.17); median kappa = 0.83). Table 2 presents the confusion matrix for the labels chosen by participants, which represents the average proportion of target and non-target labels endorsed for each expression and shows that errors were fairly consistent for each expression (e.g., incorrect judgments for fear faces were usually mislabeled as “surprise”). Table 1. Description of validity ratings for individual emotional expressions (N = 81 subjects rating 672 stimuli). Median proportion correct Mean (S.D.) proportion correct Median kappa Mean (S.D.) kappa Angry (closed) 0.90 0.84 (0.17) 0.81 0.78 (0.13) Calm (closed) 0.90 0.88 (0.07) 0.87 0.84 (0.09) Disgust (closed) 0.86 0.76 (0.23) 0.83 0.75 (0.19) Fear (closed) 0.51 0.47 (0.21) 0.58 0.54 (0.20) Happy (closed) 0.94 0.92 (0.07) 0.94 0.92 (0.06) Neutral (closed) 0.93 0.91 (0.06) 0.87 0.86 (0.08) Sad (closed) 0.91 0.83 (0.16) 0.76 0.76 (0.13) Surprised (closed) 0.61 0.61 (0.10) 0.62 0.62 (0.08) Angry (open) 0.96 0.90 (0.15) 0.92 0.88 (0.11) Calm (open) 0.81 0.79 (0.11) 0.84 0.81 (0.10) Disgust (open) 0.93 0.84 (0.21) 0.82 0.77 (0.18) Fear (open) 0.74 0.73 (0.12) 0.69 0.67 (0.12) Happy (open) 0.99 0.98 (0.02) 0.97 0.95 (0.05) Neutral (open) 0.86 0.82 (0.11) 0.86 0.83 (0.09) Sad (open) 0.59 0.60 (0.21) 0.64 0.62 (0.18) Surprised (open) 0.86 0.81 (0.13) 0.68 0.68 (0.12) Happy (open exuberant) 0.88 0.86 (0.13) 0.90 0.88 (0.09) Table options Table 2. Confusion matrix for mean (S.D.) proportion of subjects who endorsed each emotion label. Label Photograph Angry Calm/neutral Disgust Fear Happy Sad Surprised Nothing N Angry (closed) 0.84 (0.17) 0.03 (0.07) 0.05 (0.09) 0.01 (0.02) 0.00 (0.01) 0.05 (0.08) 0.00 (0.01) 0.01 (0.02) 78.93 (7.22) Angry (open) 0.90 (0.15) 0.00 (0.01) 0.05 (0.09) 0.02 (0.05) 0.00 (0.01) 0.01 (0.02) 0.00 (0.01) 0.01 (0.02) 79.07 (7.27) Calm (closed) 0.02 (0.03) 0.88 (0.07) 0.00 (0.01) 0.01 (0.01) 0.02 (0.05) 0.02 (0.05) 0.00 (0.01) 0.02 (0.02) 79.33 (7.36) Calm (open) 0.01 (0.02) 0.79 (0.11) 0.02 (0.02) 0.03 (0.05) 0.01 (0.04) 0.03 (0.05) 0.05 (0.05) 0.05 (0.04) 78.95 (7.31) Disgust (closed) 0.13 (0.18) 0.01 (0.01) 0.76 (0.23) 0.01 (0.01) 0.00 (0.01) 0.08 (0.17) 0.00 (0.01) 0.01 (0.02) 79.15 (7.47) Disgust (open) 0.03 (0.04) 0.00 (0.01) 0.84 (0.21) 0.02 (0.03) 0.00 (0.01) 0.09 (0.20) 0.01 (0.02) 0.01 (0.02) 79.14 (7.11) Fear (closed) 0.03 (0.06) 0.04 (0.06) 0.04 (0.08) 0.47 (0.21) 0.01 (0.01) 0.10 (0.16) 0.29 (0.20) 0.02 (0.02) 78.92 (7.76) Fear (open) 0.02 (0.05) 0.00 (0.01) 0.02 (0.03) 0.73 (0.12) 0.01 (0.01) 0.01 (0.03) 0.19 (0.12) 0.01 (0.01) 79.37 (7.13) Happy (closed) 0.00 (0.01) 0.06 (0.06) 0.00 (0.01) 0.00 (0.00) 0.92 (0.07) 0.00 (0.01) 0.00 (0.01) 0.00 (0.01) 79.31 (7.21) Happy (open) 0.00 (0.01) 0.01 (0.02) 0.00 (0.00) 0.00 (0.00) 0.98 (0.02) 0.00 (0.00) 0.00 (0.00) 0.00 (0.01) 79.47 (7.13) Happy (open exuberant) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.01 (0.01) 0.86 (0.13) 0.00 (0.00) 0.13 (0.12) 0.00 (0.01) 79.31 (7.20) Neutral (closed) 0.02 (0.03) 0.91 (0.06) 0.00 (0.01) 0.01 (0.01) 0.01 (0.01) 0.03 (0.05) 0.00 (0.01) 0.02 (0.02) 79.35 (7.12) Neutral (open) 0.02 (0.02) 0.82 (0.11) 0.01 (0.02) 0.04 (0.05) 0.01 (0.01) 0.03 (0.05) 0.05 (0.05) 0.03 (0.02) 79.14 (7.34) Sad (closed) 0.03 (0.05) 0.04 (0.06) 0.04 (0.07) 0.03 (0.07) 0.00 (0.01) 0.83 (0.16) 0.01 (0.04) 0.02 (0.02) 79.05 (7.17) Sad (open) 0.02 (0.04) 0.07 (0.13) 0.15 (0.14) 0.09 (0.10) 0.00 (0.01) 0.60 (0.21) 0.03 (0.03) 0.03 (0.03) 79.10 (7.33) Surprised (closed) 0.00 (0.00) 0.12 (0.05) 0.00 (0.00) 0.18 (0.06) 0.06 (0.08) 0.01 (0.01) 0.61 (0.10) 0.02 (0.02) 80.50 (0.71) Surprised (open) 0.00 (0.00) 0.01 (0.00) 0.00 (0.01) 0.14 (0.13) 0.03 (0.07) 0.00 (0.00) 0.81 (0.13) 0.00 (0.01) 79.35 (7.11) Target emotion in bold. Table options Validity ratings were similar from one actor to another. Forty-five percent (39/86) of the kappas calculated per actor ranged from 0.8 to 1.0, a range considered to reflect almost perfect concordance between the given label and intended expression by Landis and Koch (1977), and the corresponding mean proportion correct scores ranged from 0.81 to 0.93. The remaining 55% (47/86) of these kappas ranged from 0.59 and 0.79, and the mean proportion correct scores ranged from 0.67 to 0.84. There were differences in proportion correct (F(15, 435) = 24.65, P < 0.0001) and kappa scores (F(15, 435)=28.65, P < 0.0001) from one emotional expression to another indicating that some emotions were more accurately identified than others. Approximately half (8/17) of the mean kappas for emotional expression were above a 0.8 kappa cut point. The expressions above this threshold included happy (open), happy (closed), angry (open), happy (open exuberant), neutral (closed), calm (closed), neutral (open), and calm (open), and these expressions had mean proportion correct scores that ranged between 0.79 and 0.98. Another 47% (8/17) of the expressions had mean kappas between 0.6 and 0.79. These expressions included angry (closed), disgust (open), sad (closed), disgust (closed), surprised (open), fear (open), sad (open), and surprised (closed), and these expressions had mean proportion correct scores that ranged from 0.61 to 0.84. The fear (closed) expression had a mean kappa of 0.54 and a mean proportion correct score of 0.47. Table 3 shows the descriptive and inferential statistics that compare the open- and closed-mouth versions of expressions. These calculations show that expressions were not identified equally for open and closed versions. Angry, fear, and happy faces resulted in higher kappa scores with open mouths, whereas sad was more accurately identified with a closed mouth. Table 3. Comparing closed and opened mouth versions of five basic emotional expressions. Closed mean Open mean t Df Sig. (2-tailed) Kappa Angry 0.78 0.88 − 4.60 42 0.00 Disgust 0.75 0.76 − 0.25 38 0.80 Fear 0.54 0.67 − 4.12 35 0.00 Happy 0.92 0.95 − 3.13 41 0.00 Sad 0.76 0.63 4.25 40 0.00 Proportion correct Angry 0.84 0.90 − 1.90 42 0.06 Disgust 0.76 0.84 − 1.73 38 0.09 Fear 0.47 0.73 − 7.47 35 0.00 Happy 0.92 0.98 − 5.08 41 0.00 Sad 0.83 0.60 6.85 40 0.00 Table options 3.2. Reliability Reliability scores (i.e., proportion agreement) were calculated for each stimulus to quantify agreement between times 1 and 2 for each stimulus, and these values for individual stimuli can be found in Supplemental Table 4. To show the data in a succinct fashion, results are presented in aggregate for each expression (Table 4; closed and open mouth separately). Overall, there was agreement between times 1 and 2, with a mean (S.D.) reliability score of 0.84 (0.08) and median of 0.86. Table 4. Description of reliability for individual emotional expressions (N = 34 subjects, from group 2 only — rating 672 stimuli). Emotion Proportion correct Block 1 Proportion correct Block 2 Agreement between Blocks 1 and 2 Angry (closed) 0.86 0.88 0.87 Calm (closed) 0.89 0.96 0.90 Disgust (closed) 0.74 0.75 0.81 Fear (closed) 0.46 0.52 0.68 Happy (closed) 0.93 0.92 0.91 Neutral (closed) 0.92 0.99 0.94 Sad (closed) 0.84 0.84 0.85 Surprised (closed) 0.59 0.58 0.73 Angry (open) 0.91 0.92 0.90 Calm (open) 0.81 0.91 0.85 Disgust (open) 0.84 0.84 0.87 Fear (open) 0.75 0.79 0.75 Happy (open) 0.98 0.99 0.98 Neutral (open) 0.84 0.93 0.86 Sad (open) 0.62 0.66 0.77 Surprised (open) 0.83 0.84 0.80 Exuberant happy 0.88 0.86 0.87 Note — there is no model #4 or 44. Table options There was little variability in reliability scores from one actor to another (closed mouth and open mouth calculated separately). Ninety-one percent (78/86) of the actors had mean reliability scores that ranged between 0.80 and 1.00. The remaining 9% was between 0.73 and 0.79. In contrast, the mean reliability scores for each emotional expression indicate that some emotions were more reliably identified than others (see Table 4). The majority (13/17) of expressions had average reliability scores that ranged between 0.80 and 1.00, and the remaining 24% ranged between 0.68 and 0.77. 3.3. Calm and neutral validity Eighty-six proportion correct scores and kappas were calculated for the calm and neutral stimuli, one for each calm or neutral stimulus (see Table 5). Scores varied considerably from one actor to another indicating that some actors expressed calm and neutral faces better than others. Mean proportion correct for neutral faces was 0.72 (S.D. = 0.18; range: 0.12–0.91), median proportion correct was 0.76, mean kappa was 0.34 (S.D. = 0.39; range: − 0.65–0.88), and median kappa was 0.38. Mean proportion correct for calm faces was 0.56 (S.D. = 0.27; range: 0.09–0.91), median proportion correct was 0.62, mean kappa was 0.32 (S.D. = 0.40; range: − 0.65–0.88), and median kappa was 0.38. Nearly half of the neutral faces (21/43) and half of the calm faces (21/43) had kappas that exceeded 0.40, and these faces had a mean proportion correct of 0.82 and 0.79, respectively. The proportion correct scores for calm and neutral faces are positively correlated (r(41) = 0.69, P < 0.001), indicating that a model who posed an identifiable neutral face was also likely to pose an identifiable calm face. Table 5. Description of validity ratings for semi-forced choice calm vs. neutral discrimination (N = 34 subjects rating 86 stimuli). Model Calm Neutral Proportion correct Kappa Proportion correct Kappa 1 0.62 0.50 0.88 0.56 2 0.44 0.12 0.65 0.15 3 0.94 0.82 0.88 0.82 5 0.18 − 0.12 0.62 − 0.12 6 0.71 0.47 0.74 0.50 7 0.71 0.56 0.85 0.59 8 0.68 0.62 0.76 0.50 9 0.12 − 0.56 0.26 − 0.56 10 0.79 0.59 0.76 0.56 11 0.88 0.68 0.76 0.65 12 0.76 0.68 0.85 0.62 13 0.41 0.26 0.74 0.26 14 0.35 0.18 0.79 0.29 15 0.47 0.21 0.71 0.26 16 0.88 0.79 0.91 0.79 17 0.68 0.38 0.68 0.38 18 0.85 0.65 0.79 0.68 19 0.38 0.18 0.76 0.32 20 0.47 0.26 0.76 0.29 21 0.44 − 0.09 0.44 − 0.09 22 0.94 0.85 0.91 0.88 23 0.91 0.82 0.88 0.82 24 0.91 0.74 0.79 0.74 25 0.62 0.44 0.76 0.41 26 0.26 0.06 0.76 0.12 27 0.68 0.56 0.85 0.62 28 0.85 0.71 0.82 0.68 29 0.26 0.09 0.76 0.24 30 0.24 0.12 0.76 0.09 31 0.35 0.03 0.65 0.03 32 0.94 0.88 0.88 0.82 33 0.12 − 0.62 0.26 − 0.59 34 0.21 − 0.29 0.47 − 0.32 35 0.32 0.15 0.79 0.35 36 0.38 0.18 0.74 0.21 37 0.56 0.41 0.71 0.44 38 0.09 − 0.24 0.59 − 0.26 39 0.79 0.62 0.79 0.62 40 0.62 0.35 0.71 0.32 41 0.79 0.53 0.74 0.56 42 0.44 0.03 0.56 0.09 43 0.18 − 0.65 0.12 − 0.65 45 0.85 0.76 0.88 0.74 Note — there is no model #4 or 44.