یادگیری ارزیابی پیشرفت و تعیین هدف: اثر بر دستاورد خواندن، انگیزه خواندن و خودانگاره خواندن

This study investigates the effects of learning progress assessment (LPA) combined with student-set goals on students’ reading achievement, reading motivation, and reading self-concept in fourth grade. Classes (n = 41) were assigned to either an LPA group with goal setting (LPA-G), an LPA group only (LPA), or a control group (CG). Students of both LPA groups completed eight LPA tests over a period of six months, and teachers received information about their learning progress. Students in the LPA-G group specified goals before the tests and reflected their goal achievement afterwards. Results indicate that growth in reading was higher for students in the LPA group compared to students in the two other groups. Unexpected negative effects of the goal-setting procedure were found on the development of intrinsic reading motivation and individual reading self-concept. The results are discussed with regard to teacher behavior and the use of diagnostic information for instruction.

Providing teachers with diagnostic information on their students’ achievement is one basic principle to support a sensible execution of individualized instruction (Connor, Morrison, & Petrella, 2004). Moreover, diagnostic information concerning the learning progress reveals an even more continuous feedback to teachers and students. As a consequence teachers get objective information if their instruction leads to desired effects, and students see if effort and learning strategies result in an improvement of achievement. In this sense, learning progress assessment (LPA) is one prominent tool in the field of formative assessment which can serve teachers as well as students to optimize learning and instruction ( Black and Wiliam, 2009 and Clark, 2012). Reviews of the literature on effects of LPA show that this approach has a high potential to foster student learning (e.g. Stecker, Fuchs, & Fuchs, 2005). However, research has primarily focused on low-achieving students and it usually was applied to single children of a classroom. In addition, studies on LPA usually investigated teachers’ use of the diagnostic information to adapt instruction. Thus, research mainly focused on teacher behavior. However, feedback of learning progress and information about goal achievement are also key elements in self-regulated learning ( Zimmerman, 1990). Hence, LPA might be a helpful tool to support self-regulated learning when students are actively involved in LPA ( Clark, 2012). Asking students to set personal goals and reflect their goal achievement is one way to foster their involvement. While some studies investigated effects of teacher-defined goals ( Fuchs et al., 1989 and Jenkins and Terjeson, 2011), effects of student-set goals have hardly been investigated yet. Dealing with goals immediately leads to the question if providing teachers and students with diagnostic information about learning growth will have an impact on motivation and self-concept. While several studies have found positive effects of LPA on student achievement, effects of LPA on motivation and self-concept have been addressed very rarely. Taken together, the aim of our research was to evaluate effects of LPA on reading achievement, reading motivation, and reading self-concept in whole classrooms in general education. In addition, this study addresses the question if the combination of LPA and goal-setting procedures will lead to superior effects on reading and motivation. 1.1. Learning progress assessment Curriculum-Based Measurement (CBM) (Deno, 1985) is a well-established method for learning progress assessment (LPA) that provides teachers with diagnostic information on students’ learning progress. In CBM, assessment of student progress is conducted by applying parallel forms of short tests at intervals of a few days up to two weeks throughout the school year (Fuchs, 2004). Each CBM test simultaneously assesses the skills required for competent year-end performance, thus slope can be used to quantify rate of learning. Studies on the effectiveness of LPA using CBM have demonstrated that providing teachers with diagnostic information about their students’ progress leads to increased student achievement (Stecker et al., 2005). In a meta-analysis examining the results of 21 controlled studies on formative evaluation, Fuchs and Fuchs (1986) determined the average effect size to be .70 for enhanced student achievement. Some of the CBM studies investigated options to further increase effects of the CBM approach using three-group designs in which a CBM condition is compared to a CBM condition with additional support and a control group. These studies found positive effects on student achievement when teachers are supported in their instructional decision-making process (e.g. Allinder et al., 2000 and Fuchs et al., 1991). The most frequently used CBM measure to monitor student reading progress and the most researched CBM measure is oral reading fluency (ORF) (Reschly, Busch, Betts, Deno, & Long, 2009). It is defined as the number of words read aloud correctly in 1 min from a grade-level passage. ORF is hypothesized to be a higher-order skill that requires the integration of lower-level reading skills (Fuchs, Fuchs, Hosp, & Jenkins, 2001). Overall, correlations among ORF and standardized reading assessments are strong (Reschly et al., 2009). Some researchers, however, do not endorse using ORF as an indicator of reading fluency. Based on the automaticity theory for guidance (LaBerge & Samuels, 1974), Samuels (2007) argued that readers must not only identify words but concurrently need to construct their meaning to comprehend text. He emphasized this simultaneity of decoding and comprehension to be “the essential characteristic of reading fluency” (p. 564). While readers with highly automatized word recognition skills can simultaneously decode and comprehend the text, beginning readers first focus their cognitive resources on word recognition before they switch their cognitive resources to construct meaning. Riedel (2007) found that about 15% of the students were misidentified by the ORF test as good readers, when, in fact, they had poor reading comprehension. Likewise, Lerkkanen, Rasku-Puttonen, Aunola, and Nurmi (2004) identified a group of technical readers who were characterized by high levels of word reading but low levels of text comprehension. The authors note that teachers may misidentify technical readers as skilled readers because of their excellent word reading skills. Moreover, if these children do not receive special support to foster their reading comprehension, the neglect could result in a serious risk of failure when reading comprehension is needed to learn new subjects. Thus, Samuels concludes that new test-concepts are needed that require the reader to simultaneously decode and comprehend text. Shapiro, Solari and Petscher (2008) investigated the contribution of a comprehension measure in addition to ORF. They found that information about reading comprehension generally improved the prediction of students at risk for reading difficulties. In addition, their findings suggest that the addition of a comprehension measure was more essential in higher elementary grades, which should be no surprise given that readers differ in lower-level and higher-level reading processes with the latter becoming more meaningful as reading experience and reading performance increase ( Daneman, 1991). To summarize, we conclude that reading progress of all students in a classroom should be monitored to identify stagnating or regressive developments at an early stage and to evaluate whether students with different ability levels benefit from the given instruction. Furthermore, the growth rate of ORF appears to be relatively similar across students after first grade (Kim, Petscher, Schatschneider, & Foorman, 2010), but the growth rate of reading comprehension may be different across students in higher grades, and thus give valuable information for instructional modifications (Förster & Souvignier, 2011). Hence, reading progress should be assessed using differentiated measures of reading fluency and reading comprehension that provide teachers with detailed information about specific needs in the instructional decision-making process. Hierarchical models of text comprehension (Kintsch, 1998) and reading competence models (Mullis, Martin, Gonzalez, & Kennedy, 2003) may provide the theoretical basis for a new test concept. 1.2. Goal setting To decide whether instructional modifications are needed, teachers compare students’ growth rate with a goal line. Thus, learning goals play an important role in LPA (Fuchs & Fuchs, 1998). Fuchs, Fuchs et al. (1989) found that teachers who used dynamic goals employed more ambitious goals and achieved higher learning gains than teachers who used static goals. Among other aspects, more ambitious goals increase the number of instructional change prompts (Jenkins & Terjeson, 2011). Most of the studies investigating effects of LPA have focused on teacher behavior and thus examined effects of teacher-set goals. The active involvement of students in this assessment procedure has received little attention in the literature. One way to more actively involve students during LPA is to implement self-selected goal setting and reflection of goal achievement. Both strategies play an important role in self-regulation theory (Zimmerman, 1990), and might augment the learning progress. For example, Fuchs, Bahr, and Rieth (1989) found that high school students with self-selected goals improved their performance in mathematics more than students with assigned goals. Furthermore, Swain (2005) examined the effects of student-set goals in LPA of 19 students in 6th and 7th grade. Results show that significantly more students of the goal-setting group were able to state a specific reading goal than students without goal setting. However, students had difficulties setting realistic goals, indicating that they would need additional support to better understand the assessment procedure and the meaning of goals. As most research on LPA, both studies have been conducted with only few students with learning disabilities. Yet, personal goal setting has been found to be a key element in fostering achievement and motivation also for children in regular elementary schools (Rheinberg & Krug, 1999). 1.3. Reading self-concept and reading motivation Research on LPA has predominantly focused on achievement as a student outcome. Little is known about the effects of monitoring individual progress on motivational outcomes. However, given that “to become a good reader, students must possess both the skill and the will to read” (Watkins & Coffey, 2004, p. 110), it should be investigated if and to what extend motivation is affected by monitoring student progress. Following the expectancy-value theory of motivation (Wigfield & Eccles, 2000), expectancies about one’s competence (self-concept), and the value of the activity (motivation) should be considered when investigating the relationship between LPA and motivational outcomes. Reading self-concept plays a central role in reading motivation research (e.g., Chapman and Tunmer, 1997, Retelsdorf et al., 2011 and van Kraayenoord and Schneider, 1999) and has shown to be related to reading achievement (Aunola et al., 2002, Chapman and Tunmer, 1995, Chapman and Tunmer, 1997, Chapman et al., 2000 and Retelsdorf et al., 2011). The construct has been defined as the individual’s self-perception of his or her ability to read (Retelsdorf et al., 2011). However, self-perception is affected by different sources of information. Referring to the internal/external frame of reference model (Marsh, 1986), Dickhäuser (2005) found that students’ self-concept was affected differently by external and internal comparison processes. For example, self-perception of (an identical) competence can be low when others are perceived to perform very well, or it can be high when compared to one’s previous ability in that same domain. When assessing students’ self-concept, Schöne, Dickhäuser, Spinath, and Stiensmeier-Pelster (2012) suggest to differentiate between self-perceptions under the three perspectives of social, individual, and absolute norms of reference. Assessments in school usually provide information with a social frame of reference. This frame of reference might lead to low self-perceptions of ability for low-achieving students and thus to lower expectations for future success (Weiner, 1979). Providing not only teachers but also students with immediate feedback on learning progress, formative assessments address the individual frame of reference and might help students realize their progress, which in turn might positively affect their reading self-concept. One important distinction in motivation research is the difference between intrinsic and extrinsic motivation (Wigfield & Guthrie, 1997). In the area of reading, intrinsically motivated readers read because the act is experienced as inherently interesting or enjoyable, and extrinsically motivated readers read to seek external rewards. Reading motivation appears to be related to reading performance (Baker and Wigfield, 1999, Becker et al., 2010 and Guthrie et al., 2007) such that intrinsic reading motivation has been found to be more beneficial to reading development than extrinsic reading motivation (Retelsdorf et al., 2011). An empirically based theoretical framework for the study of motivation is self-determination theory (SDT; Deci and Ryan, 1985 and Deci and Ryan, 2000). Within SDT the basic psychological needs for autonomy, competence, and relatedness are postulated. In their investigation of factors that influence intrinsic motivation, Deci and Ryan (1985) argued that structures that induce feelings of competence can enhance intrinsic motivation when they are accompanied by an internal attribution of causality. Research on the effects of performance feedback on intrinsic motivation supports these assumptions (Deci, 1971, Harackiewicz, 1979 and Ryan, 1982). In a study on the effects of CBM with peer-assisted learning strategies in mathematics, Calhoon and Fuchs (2003) found that teachers and students both thought that CBM graphs increased motivation to work hard. Thus, feedback on the learning progress using formative assessments might positively influence intrinsic reading motivation when progress is attributed to ability and effort. 1.4. The present study The purpose of the current study was to evaluate the effects of LPA in general education on reading achievement, reading motivation, and reading self-concept. As outlined above, we concluded that the reading progress of all students in a classroom should be monitored. Furthermore, we argued that detailed information about reading fluency and reading comprehension is needed to enhance the instructional decision-making process. Hence, in this study on LPA, the typical CBM assessment procedure was modified in the following respects. First, we monitored the reading progress of all students in a classroom in regular education, instead of only focusing on poorly achieving students. Second, we used an extended test concept by assessing text-based and knowledge-based reading comprehension in addition to reading fluency and reading accuracy. Thus, the assessments were more closely related to theoretical models of reading competence (Förster and Souvignier, 2011 and Mullis et al., 2003). Third, to make the assessments more feasible, we reduced testing frequency to assessments taken every three weeks. In this study we address three research questions. First, is information about growth rate critical to support effective learning in addition to information about achievement? One core assumption of measuring growth is that information about differences in growth rate is critical and exceeds the information obtained by achievement status alone. Studies on the effectiveness of LPA using control group designs support this assumption (Allinder et al., 2000, Fuchs et al., 1984, Fuchs et al., 1992 and Fuchs et al., 1991). However, Schatschneider et al. (2008) used hierarchical regression techniques to evaluate the unique contribution of information about status and growth for predicting future reading achievement. Their results question the importance of slope information. In the former studies, LPA was typically compared to a control group in which teachers were not provided with any standardized diagnostic information, whereas in the study by Schatschneider et al. (2008) all teachers had diagnostic information about the learning progress. Therefore, in this study we decided to compare growth in reading achievement of classrooms using LPA to achievement gains in control classes in which teachers were provided with status information of reading performance only. Hence, the ‘net effect’ of information about growth rate compared to achievement status alone is explored. We hypothesized that information about reading growth would lead to higher gains in reading achievement than information about reading achievement status (Hypothesis 1). Second, is the combination of LPA and goal setting effective in fostering reading growth? While additional teacher support has repeatedly been found to enhance effects of LPA (Allinder et al., 2000, Fuchs et al., 1991 and Fuchs et al., 1992), we wanted to examine the effects of an active involvement of students. We expected that LPA with goal setting would lead to increased growth in reading achievement compared to LPA without goal setting (Hypothesis 2a). We consequently expected that also growth rates would be higher for the LPA group with goal setting (Hypothesis 2b). Third, does LPA with goal setting affect motivational variables? Studies that have investigated additional variables other than performance outcomes are very rare. Specifically, we investigated the effects of LPA on intrinsic and extrinsic reading motivation and on reading self-concept with regard to three different reference norms (i.e. social, individual and, absolute). Given that LPA with goal setting uses a combination of individual feedback of reading progress and internal attribution of causality we hypothesized that LPA with goal setting would positively affect students’ intrinsic but not extrinsic reading motivation (Hypothesis 3a). Likewise, we predicted that LPA with goal setting would lead to a positive change in students’ reading self-concept with regard to the individual, but not the social or absolute frame of reference (Hypothesis 3b).

3. Results 3.1. Descriptive statistics The basal reading skills of the sample were found to be typical for fourth grade students (M = 102.01, SD = 16.70). No differences between the groups existed on demographic factors such as age, gender, or language at home and their basal reading skills. Descriptive statistics for the dependent variables are shown in Table 1. At the beginning of the study, slightly more than half of the reading achievement questions were answered correctly (M = 11.79, SD = 4.63). Students of all groups rated their reading motivation to be high (Mintrinsic = 3.10, SDintrinsic = .70; Mextrinsic = 2.55, SDextrinsic = .80). Ratings of the reading self-concept were fairly high, with the highest values for the individual reference norm scale, followed by the absolute and social reference norm scales. All three groups showed growth in reading achievement and a decline in extrinsic reading motivation. 3.2. Results of the latent difference score analyses Goodness-of-fit indices of the LDS models are shown in Table 2. All LDS models show an acceptable fit to the data. Analyses of the pretest scores revealed no significant differences between the groups for all variables except extrinsic reading motivation. Students in the LPA-G group rated their extrinsic reading motivation significantly higher than students in the LPA group (z = 2.98, p = .003, see Table 3). However, given that controlling for differences in the pretest scores did not change the results, findings are reported without considering these differences. Table 2. Goodness-of-fit indices for the latent difference score models. χ2 (df) CFI/TLI RMSEA (90% CI) SRMR Reading achievement 57.71 (9) 0.98/0.96 .08 (.059, .097) .06 Reading motivation Intrinsic 75.84 (37) .99/.98 .03 (.023, .045) .04 Extrinsic 57.53 (21) .98/.98 .04 (.031, .058) .03 Reading self-concept Social 39.49 (37) 1.00/1.00 .01 (.000, .026) .03 Individual 68.25 (37) .98/.98 .03 (.019, .042) .04 Absolute 57.75 (37) .99/.99 .03 (.011, .037) .03 Note. Factor loadings and intercepts were constrained to be invariant across measurement points. χ2 = Chi-square; CFI = Comparative fit index; TLI = Tucker Lewis Index; RMSEA = Root means square error of approximation; SRMR = standardized root mean square residual. Table options Table 3. Latent difference score models for reading achievement, reading motivation and reading self-concept. Reference groupa CG LPA Predictora LPA LPA-G LPA-G Est. SEb z p Est. SEb z p Est. SEb z p Predicting initial level RA −0.07 0.06 −1.13 .258 −0.02 0.06 −0.28 .776 0.05 0.05 0.95 .345 RM (intrinsic) −0.07 0.07 −0.99 .322 −0.06 0.07 −0.97 .333 0.00 0.06 0.00 .998 RM (extrinsic) −0.09 0.05 −1.96 .050 0.05 0.05 1.02 .309 0.15 0.05 2.98 .003 RSC (social) −0.05 0.05 −1.08 .282 0.01 0.05 0.11 .916 0.06 0.05 1.08 .278 RSC (individual) 0.03 0.06 0.42 .676 0.01 0.06 0.17 .869 −0.02 0.06 −0.34 .738 RSC (absolute) −0.03 0.07 −0.32 .752 −0.01 0.05 −0.20 .840 −0.01 0.06 −0.17 .869 Predicting change RC 0.25 0.10 2.43 .015 −0.03 0.09 −0.36 .720 −0.30 0.07 −4.23 .000 RM (intrinsic) −0.03 0.06 −0.57 .567 −0.12 0.06 −2.10 .036 −0.09 0.06 −1.49 .136 RM (extrinsic) −0.03 0.06 −0.45 .655 0.04 0.05 0.77 .440 0.07 0.05 1.45 .147 RSC (social) 0.03 0.06 0.48 .634 −0.02 0.06 −0.40 .690 −0.06 0.06 −0.94 .335 RSC (individual) −0.08 0.06 −1.30 .193 −0.16 0.06 −2.91 .004 −0.09 0.07 −1.26 .207 RSC (absolute) 0.03 0.07 0.47 .640 −0.04 0.07 −0.54 .591 −0.07 0.06 −1.32 .188 Note. RA = reading achievement, RM = reading motivation, RSC = reading self-concept. Est. = standardized estimate. a Dummy coded (reference group = 0, predictor = 1). b Corrected standard errors. Bold printed estimates are significant (two-tailed). Table options 3.3. Change in reading achievement 3.3.1. Results from the pretest–posttest data For all groups, growth in reading was significantly different from zero (MΔCG = 0.39, z = 3.18, p = .001; MΔLPA = 0.75, z = 9.81, p < .001; MΔLPA-G = 0.49, z = 3.92, p < .001). As expected, LPA students showed significantly higher improvements in reading achievement than students in the CG (z = 2.43, p = .015, see Table 3). The size of the effect was dLPA-CG = 0.24. However, no significant differences in reading growth were found between the LPA-G group and the CG. A comparison of the two LPA groups showed significantly higher improvements for the LPA group compared to the LPA-G group (z = −4.23, p < .001). The effect size was dLPA-G-LPA = −0.27. 3.3.2. Results of the learning progress data For the learning progress data, both a linear and a quadratic model fitted the data reasonably well. The quadratic model (χ2 (195) = 546.626, CFI = .963, TLI = .948, RMSEA = .055, SRMR = .058) did not show a significantly better fit than the linear model (χ2 (199) = 550.637, CFI = .963, TLI = .949, RMSEA = .054, SRMR = .060). Given that also the mean of the quadratic term was not significant (M = −0.01, p > .05), results of the linear model are reported. The initial score at the first measurement point was 4.64 points. Average growth in six months was 0.25 points which equates to an effect size of d = 0.32. While the variance components of the initial level were significant (p < .001), the variance components of the slope were marginally significant (p = .060). Thus, students differed especially in their initial reading comprehension. However, there was also some interindividual variation in the growth factor. Including affiliation to the LPA-G treatment as a time-invariant predictor of intercept and slope in the model revealed that the two LPA groups did not differ in their initial reading comprehension scores (z = 0.63, p > .05). However, affiliation to the goal-setting treatment was significantly associated with reading growth (z = −2.71, p < .01). Students in the LPA group had a significantly higher average growth of 0.38 points in six months than students in the LPA-G group with 0.09 points (see Fig. 3). Accordingly, the effect size for the LPA group was considerably higher (d = .42) than for the LPA-G group (d = .10). Overall, affiliation to the goal-setting treatment explained 16% of the variance in the growth factor. Full-size image (21 K) Fig. 3. Different growth rates for the two LPA groups. LPA = learning progress assessment, LPA-G = learning progress assessment with goal setting. Figure options 3.4. Change in reading motivation Change in intrinsic reading motivation significantly differed between students in the LPA-G group and students in the CG (z = −2.10, p = .036). Students in the LPA-G group rated their intrinsic reading motivation lower at posttest, whereas ratings from the CG were higher (dLPA-G-CG = −0.16). Change scores between the two LPA groups and the LPA group and the CG were not significantly different. No significant differences were found for the latent difference scores between the three groups for extrinsic reading motivation. 3.5. Change in reading self-concept The pattern of results found for the individual reading self-concept was similar to that found for intrinsic reading motivation: A significant difference in change scores was found between the LPA-G group and the CG (z = −2.91, p = .004). Whereas ratings of the LPA-G group were lower at posttest, ratings of the CG increased. The size of the effect was dLPA-G-CG = −0.29. No significant differences in change were found between the two LPA groups or the LPA group and the CG. No significant differences were found for the latent difference scores between the three groups on the social and absolute reference norm scales.