مقایسه اجتماعی ماهی بزرگ حوض کوچک و اثرات تسلط محلی:ادغام مدل های جدید آماری، روش، طراحی، تئوری و مفاهیم اساسی

Abstract We offer new theoretical, substantive, statistical, design, and methodological insights into the seemingly paradoxical negative effects of school- and class-average achievement (ACH) on academic self-concept (ASC)—the big-fish-little-pond-effect (BFLPE; 15,356 Dutch 9th grade students from 651 classes in 95 schools). In support of the theoretical, social-comparison basis of the BFLPE, controlling for direct measures of social comparison (subjective ranking of how students compare with other students in their own class) substantially reduces the BFLPE. Based on new (latent three-level) statistical models and theoretical predictions integrating BFLPEs and ‘local dominance’ effects, significantly negative BFLPEs at the school level are largely eliminated, absorbed into even larger BFLPEs at the class level. Students accurately perceive large ACH differences between different classes within their school and across different schools. However, consistent with local dominance, ASCs are largely determined by comparisons with students in their own class, not objective or subjective comparisons with other classes or schools. At the individual student level, ASC is more highly related to class marks (from report cards) than standardized test scores, but the negative BFLPE is largely a function of class-average test scores. Consistent with theoretical predictions, BFLPEs generalize across objective and subjective measures of individual ACH, and BFLPEs are similar for the brightest and weakest students

1. Introduction In education settings, a positive academic self-concept (ASC) is both a highly desirable goal and a means of facilitating subsequent academic achievement (ACH), academic accomplishments, and educational choice behaviors including subject choice, coursework selection, academic persistence, and long-term educational attainment (e.g., Chen et al., 2013, Guay et al., 2004, Marsh, 1991 and Pinxten et al., 2010). In the formation of ASC, individuals must juxtapose their perceived accomplishments and appropriate standards or frames of reference for evaluating their accomplishments. The same objective characteristics and accomplishments can lead to disparate ASCs, depending on the frames of reference that individuals use to evaluate themselves, and these self-beliefs have important implications for future choices, performance, and behaviors (Marsh, 2007; also see William James, 1890/1963, p. 310). Following from this tradition, we extend research on the big-fish-little-pond-effect (BFLPE), which describes the seemingly paradoxical negative effects of school- and class-average ACH on academic self-concept (Marsh et al., 2008 and Marsh and Parker, 1984). Thus, students in academically selective settings compare their own accomplishments with those of their classmates and form less positive academic self-concepts than if they were in less academically selective settings. The BFLPE has important theoretical implications for social comparison theory, important methodological implications for more appropriate multilevel models of contextual effects, and important policy/practice implications related to the unintended negative consequences of selective educational settings (Marsh et al., 2008). Hence, the present investigation is a substantive-methodological synergy (Marsh & Hau, 2007), an integration of new and evolving methodology to address new substantive issues with theoretical and policy implications. Here we combine new statistical models, methodology, theoretical perspectives, study design and data to test new BFLPE predictions and contribute to the resolution of unresolved theoretical and substantive issues. Statistically, we introduce the first BFLPE application of latent three-level models (students nested within classes, classes nested within schools) that integrates the advantages of confirmatory factor analysis (CFA), structural equation models (SEM), and multilevel models. This allows us to juxtapose the separate and combined effects of class-average and school-average effects on ASC that are typically confounded. It also allows us to test new BFLPE theoretical predictions based on the integration of social comparison theory (Festinger, 1954 and Marsh et al., 2008) and the local dominance effect that posits that students use the most local frame of reference even when they know that it is not representative and more appropriate normative information is available (Alicke et al., 2010 and Zell and Alicke, 2009). Theoretically, we extend direct measures of social comparisons (individual student rankings of their ability in relation to their class) proposed by Huguet et al. (2009) to include direct social comparison measures of the class and school. We integrate these new and extended measures with improved statistical models to test novel theoretical predictions about how social comparison, local dominance, and grading-on-a-curve (the tendency for teachers to give a similar distribution of school marks, independent of the absolute ability level of students in their class) processes underpin the BFLPE in relation to alternative frames of reference. Combining these new approaches, we evaluate and juxtapose conflicting predictions about how the BFLPE is moderated by individual student ACH (objective and subjective) – whether the negative effects of class- and school-average ACH on ASC differ for the brightest and weakest students.

5. Results 5.1. Relations among constructs: Construct validity of ASCs and social comparison rating We begin with an evaluation of relations among constructs (see Appendix) in support of the convergent and discriminant validity of the ASC responses, but also as an advanced organizer to subsequent analyses. The ASC factors are remarkably distinct. Indeed, the average correlation among the three latent ASC factors is close to zero. Although there is a small positive correlation between the two language ASCs (DSC and ESC; r = .190), both these ASCs are somewhat negatively correlated with MSC. Each of the ASCs is significantly correlated with the matching ACH based on both class marks and test scores, but correlations with class marks are much higher than those based on test scores. These patterns of results, the extreme domain specificity of ASCs in different school subjects in relation to each other and to measures of ACH, and the consistently higher correlations with class marks than test scores, are consistent with previous research and a priori predictions. The results provide strong support for both the convergent and discriminant validity of the ASC responses. Support for the domain specificity of the class marks and particularly test scores in the different school subjects is much weaker (i.e., the correlations between different domains are much higher) than for the ASC responses. A unique feature of the present investigation is the addition of the direct social comparison measures that are important in testing processes underlying the BFLPE, but are also of interest in their own right. Indeed, the domain specificity – both the size and pattern – for relations of individual comparisons (meVclass) is similar to those observed for the ASC factors and substantially less than correlations among class marks and particularly test scores. Furthermore, the correlations between these single-item meVclass ratings and matching latent ASC factors are very high (.706–.854, see Appendix). This is consistent with the local dominance proposal and social comparison processes posited as a basis of the BFLPE: each of the ASCs reflects primarily how students compare with other students in their class. Also of interest are social comparison ratings by students in relation to a global academic construct as to how their class compared to other classes in the school (classVschool), and how their school compared to other schools in the country (schoolVcountry). Although these ratings are domain-general rather than domain specific, there is support for their construct validity. The individual student classVschool ratings are significantly correlated with class-average test scores and individual student schoolVcountry ratings are significantly correlated with school-average test scores (see Appendix). These results suggest that students have reasonably accurate perceptions of how the average ACH levels of students in their class compares with other classes in their school, and how the average ACH level of students in their school compares with other schools in their country. 5.2. BFLPE at school and class levels: Juxtaposition of two- and three-level models (Research hypotheses 1 and 2) Previous latent variable multilevel studies of the BFLPE have been two-level models with individual students nested within either classrooms or schools. Hence, an important contribution of the present investigation is the juxtaposition of two-level models (classroom BFLPEs or school-level BFLPEs), like those considered in previous research, with those based on the more appropriate three-level model that incorporates both class- and school-levels that are also relevant to tests of the local dominance effect. In the two-level model with school-average test scores (BFML1A, Table 1; ignoring class-average test scores), as in most BFLPE studies, there is clear support for the BFLPE. The school-average test score in each subject has a statistically significant negative effect on the corresponding ASC (BFLPEs = −.094 to −.196). In the two-level model with class-average test scores (BFML1B; ignoring school-average test scores), support for the BFLPE is substantially stronger (BFLPEs = −.234 to −.407). For the more appropriate three-level model (BFML1C), the BFLPE at the school level is completely eliminated after controlling for the effects of class-average ACH – all the effects are close to zero and none is statistically significant (i.e., the credibility interval around the estimate contained 0). Thus, consistent with the local dominance effect, the smaller BFLPEs associated with school-average ACH were completely accounted for by the more local classroom context so that there were no additional effects of school-average ACH beyond what could be explained in terms of the school. This might even suggest that school context really has no effect and its apparent effect is merely a reflection that schools with high school-average achievement are made up of classes with high class-average achievement. However, the direction, significance, and even sizes of the BFLPEs at the class-level are similar to those in the corresponding two-level model based on students and classes (BFLPEs = −.209 to −.426; model BFML1C in Table 1). Furthermore, our results suggest that studies based on negative effects of school-average ACH on ASC probably underestimate the size of the BFLPEs when based on the more appropriate class-average ACH. Table 1. Four BFLPE Models (with and without class marks) predicting academic self-concept factors (Dutch, English, and Mathematics) based on various combinations of student (L1), class-average (L2) and school-average (L3) achievement and class marks. BFLPEs shaded in gray: Standardized path coefficients (Standard Errors). Basic BFLPE models Without class marks With class marks BFML1A BFML1B BFML1C BFML2 Individual student level (L1) Dutch SC predictors L1 Dutch test .289(.011)a .345(.012)a .348(.013)a .198(.012)a L1 Dutch class marks .654(.009)a English SC predictors L1 English test .373(.011)a .448(.012)a .451(.013)a .210(.010)a L1 English class marks .732(.007)a Math SC predictors L1 Math test .397(.010)a .569(.013)a .569(.013)a .276(.010)a L1 Math class marks .748(.007)a Classroom level (L2)a Dutch SC predictors L2 Dutch test −.271(.024)a −.221(.028)a −.186(.024)a English SC predictors L2 English test −.234(.021)a −.209(.022)a −.127(.016)a Math SC predictors L2 Math test −.407(.020)a −.426(.025)a −.242(.019)a School level (L3)a Dutch SC predictors L3 Dutch test −.196(.041)a −.066(.043) −.012(.032) English SC predictors L3 English test −.094(.029)a −.021(.031) .006(.020) Math SC predictors L3 Math test −.159(.034)a .064(.038) .052(.024) Note. The four columns each represent a different three-level model in which the dependent variables were self-concepts (SCs) in Dutch, English, and mathematics. Predictor variables are test scores at student (L1), class (L2), and school (L3) levels, and class marks. Class (L2) and School (L3) level test scores are aggregates of student (L1) test scores. Big-fish-little-pond effects shaded in gray: effects of class-average and school-average achievement on self-concept. Values in parentheses are posteriori SDs which are similar to standard errors (i.e., SDs of a sampling distribution). Ns are 15,356 students, 651 classes, and 95 schools. a 95% critical intervals of the parameter estimate do not contain the value of zero. Table options 5.3. Juxtaposition of class marks and test scores as measures of L1 student achievement (Research hypothesis 3) Previous research (e.g., Marsh, 2007; also see earlier discussion of correlations in Appendix) has consistently shown that class marks are a stronger predictor of ASC than standardized test scores. Indeed, class marks are a more transparent and local source of feedback to students about their accomplishments within the context of a particular class. However, class marks are typically not considered in BFLPE studies because they are usually idiosyncratic to individual teachers and do not provide a common metric across all schools and classes (one of the requirements of BFLPE studies). This is due in part to the grading-on-a-curve effect (i.e., the best and worst performing students in any particular class get the highest grades and lowest grades respectively, largely independent of the average ability of the class as a whole). Indeed, as noted earlier, grading-on-a-curve effects contribute to the BFLPE so that it is important to evaluate the inclusion of class marks in our BFLPE models. In Model BFML2 (Table 1) class marks were added to Model BFML1C (Table 1). Consistent with predictions, class marks are strongly related to the corresponding ASCs. Indeed, when class marks are added to the model, the effects of L1 test scores on ASCs are substantially reduced – but still positive and highly significant. Of particular relevance, and consistent with a priori predictions, the negative effects of L2 test scores on ASC (the BFLPE) are also reduced by the inclusion of L1 class marks – but still negative and highly significant. This suggests a complex pattern of results in which ASCs are more strongly related to class marks than standardized test scores, and that the grading-on-a-curve effect is one of the processes that leads to the BFLPE (i.e., class marks mediate the effects of test scores on ASCs). 5.4. Direct social comparison ratings as predictors and outcomes: Students, classes, and schools (see Research hypothesis 4) We asked students how they compared to other students in their class (meVclass in mathematics, Dutch, English, global academic). Here we add these comparative ratings to models of the BFLPE based on ASC latent factors to test predictions that the BFLPE is based substantially on social comparison processes. In model BFCMP1 (Table 2), we added the meVclass social comparison ratings to the prediction of each of the ASC latent factors (Model BFML1C in Table 1). Consistent with predictions, the inclusion of the meVclass ratings substantially reduced the positive effects of individual student (L1) test scores and substantially reduced the negative effects of class-average (L2) test scores, but had little effect on small, non-significant effects of school-average (L3) test scores. However, the positive effect of L1 test scores remained significantly positive for all three subjects, and the negative effects of class-average effects remained significantly negative for two of the three subjects (the BFLPE was non-significant for English). Table 2. Two BFLPE Models (with and without class marks) predicting Academic Self-Concept factors (Dutch, English, and Mathematics) based on student (L1), class-average (L2) and school-average (L3) achievement, and meVclass ratings. BFLPEs shaded in gray: Standardized Path Coefficients (Standard Errors). BFLPE models Without class marks With class marks Model BFCMP1 Model BFCMP2 Individual student level (L1) Dutch SC predictors L1 Dutch test 154(.010)a .111(.010)a L1 Dutch class marks .363(.008)a Dutch MeVclass (MvC) 638(.007)a .512(.008)a English SC predictors L1 English test 128(.010)a .085(.010)a L1 English class marks .358(.007)a English MeVclass (MvC) 790(.006)a .605(.006)a Math SC predictors L1 Math test 150(.009)a .121(.008)a L1 Math class marks .326(.007)a Math MeVclass (MvC) .814(.006)a .631(.007)a Classroom level (L2) Dutch SC predictors L2 Dutch test −.101(.025)a −.118(.020)a English SC predictors L2 English test −.002(.017) −.014(.015) Math SC predictors L2 Math test −.096(.019)a −.098(.013)a Dutch SC predictors L3 Dutch test −.042(.033)a −.008(.026)a English SC predictors L3 English test −.023(.033) −.010(.016) Math SC predictors L3 Math test .029(.025) .037(.018) Note. The two columns represent different three-level models in which the dependent variables are self-concepts (SCs) in Dutch, English, and mathematics. Predictor variables are test scores at student (L1), class (L2), and school (L3) level test scores, class marks, and comparative ratings of student perceptions of how they compare with other students in their Dutch, English, and mathematics class (meVclass). Class (L2), and School (L3) level test scores are aggregates of student (L1) test scores. The two models differ only in the inclusion or not of class marks. The Big-fish-little-pond effects shaded in gray: effects of class-average and school-average achievement on self-concept. These two models differ from corresponding models BFML1C and BFML2 in Table 1 only in the inclusion of meVclass ratings. Also see additional models with classVschool and schoolVnation comparative ratings in Supplemental Materials. Values in parentheses are posteriori SDs which are similar to standard errors (i.e., SDs of a sampling distribution). Ns are 15,356 students, 651 classes, and 95 schools. a 95% critical intervals of the parameter estimate do not contain the value of zero. Table options Next we added class marks to the prediction of each ASC factor (Model BFCMP2; the corresponding Model without meVclass ratings is Model BFML2 in Table 1). The addition of class marks further reduced the positive effects of individual student test scores, but they remained significantly positive. The addition of class marks also reduced the positive effects of the meVclass ratings, but the meVclass effects were still systematically larger than the effects of class marks. It is also interesting to note that the effects of class marks are substantially smaller in Model MFCMP2 (Table 2) that includes meVclass ratings than the corresponding Model BFML2 (Table 1) that does not. However, the inclusion of class marks had essentially no effect on BFLPEs based on class-average test scores. The effect of class-average tests scores remained statistically significant for two subjects and non-significant for one (English). It is also important to note that for models without meVclass ratings (Table 1), the inclusion of class marks substantially reduced the size of the BFLPE – consistent with a grading-on-a-curve effect. However, for models that already included meVclass effects (Table 2), the inclusion of class marks did not reduce the BFLPE at all (there were actually small, non-significant increases in the BFLPEs). Hence, controlling for social comparison effects operationalized by meVclass ratings eliminated the grading-on-a-curve effect that has been such a salient component of the BFLPE. However, this should not be surprising as meVclass and class marks are strongly related – both logically and empirically (see Appendix). In supplemental analyses (see Section 6, Table 5 in the Supplemental Materials), the inclusion of classVschool and schoolVnation ratings to Model BFCMP2 were small and had no effect on the BFLPE based on class-average test scores. Thus, even though students knew how their classes compared with other classes in the same school and knew how their school compared with other Dutch schools, their inclusion did not affect the BFLPE. In these supplemental analyses we also tested the BFLPE based on responses to these single-item meVclass ratings as outcomes (instead of latent ASCs as outcomes), and compared these results based on parallel models with ASCs as outcomes. The pattern of results and even the relative size of the effects of results based on the latent ASC factors were captured by the results based on single-item meVclass ratings (see Section 6, Table 4 in Supplemental Materials). These results are consistent with the claim that the BFLPE is largely based on social comparison processes that occur at the local classroom level. 5.5. Does the size of the BFLPE differ for more and less able students? (Research hypothesis 5) As noted earlier, there is a theoretical debate surrounding this empirical question – whether the brightest and weakest students suffer larger or smaller BFLPEs – that has important implications for understanding the BFLPE and countering its negative effects. In the first of our final set of models (Model BFINT1 in Table 3) we evaluate whether the BFLPE varies with individual student ACH, operationalized as the interaction between individual and class-average test scores for Dutch, English, and mathematics. Consistent with a priori theoretical predictions and previous research, the three interaction effects are all very small and not even consistent in direction. For DSC the interaction is significantly positive but very small (.033) in relation to the large negative BFLPE (−.192) so that even the highest achieving students suffer substantial BFLPEs. However, the interaction effect is non-significantly positive for ESC and non-significantly negative for MSC. Hence, the BFLPE is quite robust in relation to the objective measures of individual ACH; high and low achieving students all experience the BFLPE. Table 3. Three BFLPE Interaction (BFInt) models predicting Academic Self-Concept factors (Dutch, English, and Mathematics) based on: Student (L1), class-average (L2) and school-average (L3) achievement; meVclass, classVschool, and SchoolVnation comparative ratings; and interactions between L2 test scores and either L1 test scores or L1 meVclass comparative ratings. BFLPEs shaded in gray. standardized path coefficients (Standard Errors). BFLPE models BFINT1 BFINT BFINT3 Individual student level (L1) Dutch SC predictors L1 Dutch test .210(.012)a .107(.010)a .112(.010)a L1 Dutch class marks .652(.009)a .360(.009)a .358(.009)a Dutch MeVclass (MvC) .511(.008)a .511(.008)a Dutch ClassVschool(CvS) .036(.008)a .037(.008)a Dutch SchoolVnation(SvN) .027(.008)a .029(.008)a Dutch L1Ach × L2 Dutch test .033(.011)a .000(.010) Dutch MvC × L2 Dutch test .013(.009) English SC predictors L1 English test .218(.009)a .087(.009)a .082(.010)a L1 English class marks .748(.007)a .357(.007)a .358(.007)a English MeVclass (MvC) .605(.006)a .606(.006)a English ClassVschool(CvS) .019(.006)a .018(.006)a English SchoolVnation(SvN) .010(.006) .010(.006) L1 English Ach × L2 English test .021(.011) .007(.009) English MvC × L2 English test −.027(.008)a Math SC predictors L1 Math test .274(.009)a .117(.009)a .118(.008)a L1 Math class marks .748(.007)a .327(.007)a .326(.007)a Math MeVclass (MvC) .631(.007)a .632(.007)a Math ClassVschool(CvS) .010(.005) .009(.005) Math SchoolVnation(SvN) .012(.006) .011(.005) L1 Math Ach × L2 Math test −.009(.008) −.017(.007)a Math MvC × L2 Math test −.018(.006)a Classroom level (L2) Dutch SC predictors L2 Dutch test −.192(.022)a −.127(.019)a −.131(.019)a English SC predictors L2 English test −.126(.019)a −.019(.015) −.018(.018) Math SC predictors L2 Math test −.243(.017)a −.103(.015)a −.101(.011)a School level (L3) Dutch SC predictors L3 Dutch test −.001(.031) −.008(.026)a −.010(.026) English SC predictors L3 English test .010(.021) −.007(.017) −.011(.017) Math SC predictors L3 Math test .048(.026) .033(.019) .040(.017) Note. The three columns represent different three-levels models in which the dependent variables are self-concepts (SCs) in Dutch, English, and mathematics. Predictor variables are test scores at student (L1), class (L2), and school (L3) level test scores, class marks, comparative (meVclass, classVschool, and schoolVnation) ratings, and the interaction between L2 Test scores and either L1 Achievement or L1 meVclass comparative ratings. Class (L2), and School (L3) level test scores are aggregates of student (L1) test scores. The two models differ only in the inclusion or not of class marks. The BFLPEs are shaded in gray - effects of class-average (L1) and school-average (L2) achievement on self-concept. Also see additional models with classVschool and schoolVnation comparative ratings in Supplemental Materials. Values in parentheses are posteriori SDs which are similar to standard errors (i.e., SDs of a sampling distribution). Ns are 15,356 students, 651 classes, and 95 schools. a 95% critical intervals of the parameter estimate do not contain the value of zero. Table options The problem with model BFINT1 (Table 3), like most research on this issue, is that individual student ACH is operationalized in relation to objective measures of ACH. Here we extend these analyses, operationalizing ACH in relation to each student's own comparative rating of their own ACH relative to other students in their class (meVclass), school (classVschool), and country (schoolVnation). In Model BFINT2, we simply add these three comparison ratings to model BFINT1. As already demonstrated, meVclass ratings are substantially related to ASCs and controlling meVclass ratings leads to a smaller BFLPE (consistent with the social comparison basis of the BFLPE). What is critical here is that the interactions between individual student and class-average test scores are not significantly positive for any of the ASCs. Indeed, although very small, the interaction is significantly negative for math (−.017). In Model BFINT3, the interaction term is defined by the relation between meVclass ratings (rather than test scores) and class-average ACH. Theoretically this is important because meVclass ratings and the associated interaction terms are based on each student's own subjective perspective of their ACH rather than objective test scores. Furthermore, it shifts the research question from whether the BFLPE varies with individual ACH in general to whether the BFLPE varies for each student's perceived ACH relative to the student's actual class. However, again there are no significantly positive interactions; small interactions are significantly negative for ESC (−.027) and MSC (−.018). In supplemental analyses (Supplemental Materials, Section 9, Interaction Effects) we further tested local dominance effect predictions, asking whether the size of the BFLPE varies with individual student perceptions of how their class compares with other classes and how their school compares with other schools. This was operationalized by adding new terms for the interaction of class-average ACH with classVschool and schoolVnation ratings. However, neither of these new interaction terms is statistically significant for any of the three ASCs. In strong support for the predictions based on the local dominance effect, the BFLPE does not vary as a function of student perceptions of how their class compares with other classes in the school or how their school compares with other schools in the country.