استفاده از ارزیابی مداخله گرا برای تشخیص و تصورات غلط تغییرات آب و هوایی دانش آموزان: مطالعه موردی سنگاپور

The evaluation of classroom-based educational interventions is fraught with tensions, the most critical of which is choosing between focusing the inquiry on measuring the effects of treatment or in proximately utilizing the data to improve practice. This paper attempted to achieve both goals through the use of intervention-oriented evaluation of a professional development program intended to diagnose and correct students’ misconceptions of climate change. Data was gathered, monitored and analyzed in three stages of a time-series design: the baseline, treatment and follow-up stages. The evaluation itself was the ‘intervention’ such that the data was allowed to ‘contaminate’ the treatment. This was achieved through giving the teacher unimpeded access to the collected information and to introduce midcourse corrections as she saw fit to her instruction. Results showed a significant development in students’ conceptual understanding only after the teacher's decision to use direct and explicit refutation of misconceptions. Due to the accessibility of feedback, it was possible to locate specifically at which point in the process that the intervention was most effective. The efficacy of the intervention was then measured through comparing the scores across the three research stages. The inclusion of a comparison group to the design is recommended for future studies.

Feedback is an invaluable resource in program planning, implementation and review. In the classroom, feedback offers a window for teachers to objectively assess the impact of an instruction on learning. Gathering feedback for teaching and learning program evaluation is usually implemented through formative assessment administered in the form of brief tests or short tasks, which provide actual data on the progress of learning and serve as prompts for immediate and timely refinement to the instructional approach (William & Thompson, 2009). Collecting and incorporating responses for in situ teaching and learning intervention through traditional research methods pose several limitations. For instance, comparison groups may not always be available for the implementation of an experimental design. In addition, randomization is an issue that is often controversial. Indeed, the random assignment of students to control and experimental groups raises ethical questions and a practice that may not always be welcomed by teachers, school boards and parents (Check & Schutt, 2012; Monk, 1984). Conventional designs also disallow the contamination of treatment with the effects of treatment to ensure data fidelity, essentially blocking feedback from informing the program of intervention. This practice, however, does not optimize the data's latent capability to positively influence practice. In this article, the authors explored the usefulness of intervention-oriented evaluation as a tool for classroom-based research cum professional development activity with teachers. The topic of interest was climate change, a subject whose tendency to be misunderstood by learners is well documented in the literature as well as in previous researches by the authors (Chang & Pascua, 2014). The intervention and evaluation designs were heavily adapted to the unique situations in the school setting, specifically the unavailability of a comparison group and the emphasis on self-reflection by the stakeholders as a key feature of the activity. Through a quasi-experiment, the evaluation aimed to consciously ‘contaminate’ the data, and successively incorporated corrections introduced to the overall analysis. With the aid of time-series design, the authors endeavored to expand discussion on alternative methods of formative evaluation whose flexibility allows for non-conventional techniques that combine usefulness with reliable empirics without the need to sacrifice one for the other. Ultimately, the project intends to put forward recommendations for curriculum and pedagogical improvement for climate change education in Singapore. 1.1. Intervention-oriented evaluation The conceptual and operational design of the activity followed Patton's (2008) concept of intervention-oriented evaluation, a sub-type of utilization-focused evaluation, which posits the evaluation process as a built-in instead of an add-on feature. With utility of evaluation as its core focus, the approach highlights the participation of intended users in the entire process of evaluation. In this study, the evaluation itself functioned as the ‘intervention’ such that the data was allowed to ‘contaminate’ the treatment. This was achieved through giving the teacher unimpeded access to the collected information and to introduce midcourse corrections as she saw fit to her instruction. Thus, the evaluation project was situational and indeed, even personal, in an environment of continuing cycle of reflection and innovation that allowed the data to show both the strengths and weaknesses of the intervention. Flowers (2010) commends this process for its inclusive approach which increases the sense of ownership and sustained involvement of participants. 1.2. Diagnosis and refutation of misconceptions With the evaluation's operant design, it was necessary for a methodological approach that sanctions for repeated, unobtrusive testing and one that accommodates midcourse changes. Time-series was the favored approach to carry out the objectives of the research with data from the series made available for the teacher's perusal. Popularized by Gottman, McFall, and Barnett (1969), the time-series design is ideal for planned interventions when a control group is unavailable or when the parameters of an experimental design such as random assignment are not met. With only a few participants subjected to frequent and regular data collection, the method allows for the compilation of a wide variety of information on variables that interact in a research environment. Unlike the conventional pre-test and post-test approach, the time-series uses multiple baselines to establish pre-treatment conditions, which then becomes the basis of analysis on the degree of change (Check & Schutt, 2012). Earlier papers in education research by Mayer (Farnsworth and Mayer, 1984, Mayer and Kozlow, 1980, Mayer and Lewis, 1979 and Mayer and Rojas, 1982) delved into the development of the single-case time-series design in measuring student achievement and attitude. Its usefulness had also been recognized in special education studies (Horner et al., 2005 and Marston, 1988), in measuring teaching effectiveness (Lin & Lawrenz, 1999), and self-regulated learning (Schmitz & Wiese, 2006). Time-series is preferred in longitudinal inquiries such as in the assessment of the effectiveness of school reforms (Bloom, 2003 and May and Supovitz, 2006), the effect of financial aid to educational quality (Henry & Rubenstein, 2002) and the impact of lotteries to state expenditure for education (Moon, Stanley, & Shin, 2006). Ortega and Iberri-Shea (2005) posit that time-series “constitutes the single best formal strategy for investigating effects of instruction longitudinally” (p. 33) in their meta-analysis of approaches used for second-language acquisition. Despite its recognized usefulness, however, studies using the design remain few and far between when compared with the more popular pre-test and post-test approach. 1.3. Misconceptions and climate change Climate change remains a misunderstood phenomenon despite its prominence in social and political discourses and its inclusion in various courses within environmental and geographic education. In fact, the literature attests to the prevalence of misconceptions in common knowledge about climate change causes, processes and impact across research contexts. This chronic issue is attributed to the prevalence of misconceptions in learners’ mental schema. With a topic that is inherently complex and non-intuitive (McCaffrey & Buhr, 2008) the challenge in teaching climate change lies in on how to integrate misconceptions in mainstream pedagogical practices. Indeed, it had been reiterated that misconceptions bar efforts at improved climate literacy (Harrington, 2008 and Dupigny-Giroux, 2010). In context, this study in Singapore was conducted within a social backdrop of a government that has for the past decade dramatically increased its efforts to draw its citizenry's attention to climate change, its mitigation and related adaptation measures. Through the Singapore Green Plan (Ministry of Environment and Water Resources [MEWR], 2006), the state overtly communicated its acknowledgment of rising global temperature and its inherent threat to the island-nation's hard-earned development gains, its geography as well as resource vulnerabilities (Chang, 2014). A study by the National Climate Change Secretariat (Cheam, 2012) shows that many Singaporeans are keen to understand climate change in much more depth, and one suggestion put forward by the respondents was the inclusion of climate change in school curriculum. At present, climate change as a theme under environmental education is interspersed in Science and Social Studies, with Geography as the lead discipline in examining the world's climate system at the Secondary school level (Chang, 2014). In 2012, a joint syllabus revision exercise was conducted by the National Environment Agency and the Ministry of Education in which it was highlighted that environmental issues, particularly climate change, will feature more prominently in Secondary level curriculum (Cheam, 2012). Nonetheless, while the classroom figures as a key venue for environmental learning, young people in Singapore regard the media, particularly the Internet and television as the most ubiquitous sources of information on climate change (Chang, 2014 and Tan, 2013). Earlier research by the authors established baseline understanding on the lack of depth in the understanding of climate change by Singapore students (Chang and Pascua, 2014 and Chang and Pascua, in press). In addition, the misconceptions detected reflected to a great extent common confusions indicated in the international literature. Such include the confusion of the greenhouse effect with ozone layer depletion (Hansen, 2010, Lee et al., 2007, Österlind, 2005 and Rebich and Gautier, 2005), the types and properties of greenhouse gases and the misunderstanding of the source, type and nature of the heat that is trapped (Boon, 2010, Gautier et al., 2006, Hansen, 2010, Mower, 2012, Österlind, 2005, Punter et al., 2011, Rebich and Gautier, 2005 and Shepardson et al., 2009). Confusions as to effects and management of the phenomenon are also prevalent. For instance, climate change is commonly perceived as related to certain environmental issues such as lead pollution, radioactive contamination and acid precipitation (Mower, 2012, Papadimitriou, 2004 and Punter et al., 2011). The reviewed literature is also denotative of the persistence of misconceptions prior to formal instruction and their tendency to obstruct proper learning (Duit & Treagust, 2003). Commonly referred to as misconceptions, these belief systems are formed from various sources of misinformation such as the media (Hansen, 2010) and weak instruction (Boon, 2010, Cordero et al., 2008 and Lambert et al., 2012). Misconceptions are not easy to detect (Chi & Roscoe, 2002) and efforts at implementing effective intervention have been documented to be particularly difficult and often unsuccessful (Limón, 2001). This is especially true when misconceptions are deeply rooted in the daily life experiences of learners (Duit, 1999), such as through repeated exposure to unsubstantiated information on the Internet. Studies that have reported on interventions to correct misconceptions concur on the seeming permanence of misconceptions. In their evaluation of the degree to which misconceptions are refuted in students’ understanding of the greenhouse effect, Gautier et al. (2006) remarked on the resilience of misconceptions against instruction. Similarly, Lee et al.’s (2007) classroom intervention with elementary students yielded only partial improvement in conceptual knowledge with majority of misconceptions retained. The tenacity of misconceptions is, to a degree, dependent on the nature of instruction. Often, instruction fails to consider for misconceptions and faulty linkages between propositions, resulting in a learner simply appending new information to an already existing body of misconceptions (Chi, 2008 and Chi and Roscoe, 2002). In this light, this study aimed to simultaneously implement an intervention and evaluation aimed at determining pedagogical approaches that aid in the refutation of climate change misconceptions. The questions that guided the inquiry were: (1) Why are climate change misconceptions resistant to instruction? (2) What type of refutational instruction effectively corrects misconceptions? (3) To what extent does instruction for refutation result in partial and/or full revision of misconceptions? 2. Method The research and evaluation was a collaborative professional learning activity with Geography teachers of a girls’ school in Singapore. The intervention was conducted with a class of 31 Secondary 4 (Grade 9) students who were doing back classes for several units in Secondary 3 Geography, including the topic of climate change. Ms Leong (pseudonym), an experienced Geography teacher, taught the class. The key criterion for selecting this site was the teacher's and the department's keenness to participate in what they saw was a partnership to help students learn better. While Ms Leong had sole involvement in the activity, it was agreed that a review would be conducted with the department after its conclusion.