اثرات مدیریت مدرسه محور: منابع و یا حاکمیت تغییراتی؟ مدارک و شواهد از مکزیک

In their struggle to improve student learning, many developing countries are introducing school-based management (SBM) reforms that provide cash-grants to school councils. School councils are expected to work collaboratively and decide on the best use of the funds. In this paper, we study the effects of one such program in Mexico on student outcomes. We complement the differences-in-differences analysis by qualitatively exploring program implementation. Results suggest the program had substantial positive effects on third grade Spanish test scores, with most benefits accruing to schools receiving SBM cash grants for the first time. These results are robust to alternative model specifications. The implementation analysis suggests school councils did monitor grant use, but parental participation did not significantly improve in other areas. Our findings suggest that the observed positive program effects are likely to be the result of providing schools with financial resources to meet pressing equipment, material, and infrastructure needs.

Despite great improvement in education access in recent decades, developing countries continue to struggle to ensure that children in primary school acquire the skills and knowledge necessary to further their education and pursue better economic opportunities. Toward this end, many developing countries have implemented school-based management (SBM) reforms. SBM has been championed by international organizations such as The World Bank as a promising way to improve education through better use of financial resources (World Bank, 2003). It is argued that by decentralizing decision-making authority and responsibility for school operations from the federal level to local stakeholders, these decisions can better reflect local needs and priorities leading to improved student outcomes (Barrera-Osorio et al., 2009, King and Ozler, 1998 and Malen et al., 1990; Khattri et al., 2010). There are currently estimated to be more than 800 SBM programs around the world operating in countries as diverse as El Salvador, New Zealand, Hong Kong, Kenya, and Israel (Barrera-Osorio et al., 2009). The popularity of SBM programs notwithstanding, rigorous evidence supporting the impact of such programs on student achievement is only beginning to emerge. In their review of evaluations of SBM programs in over 20 countries, Barrera-Osorio and his colleagues (2009) found mixed evidence regarding their impact on student test scores. Several cases showed positive results, such as programs in El Salvador, the Philippines, Indonesia, and Nicaragua (King and Ozler, 1998, Jimenez and Sawada, 2003 and Sawada and Ragatz, 2005; Kattri, Ling & Jha, 2010). In addition, several studies showed improvement on pass rates and, to a lesser degree, on dropout rates (Paes de Barros & Mendonca, 1998; Jimenez & Sawada, 2003; di Gropello & Marshall, 2005; Gertler, Patrinos, & Rubio Codina, 2006; Skoufias and Shapiro, 2006 and Carnoy et al., 2008). Other studies found no effects of program participation on student achievement (See for example Carnoy et al., 2008 for Brazil, or BN Consult & Marshall, 2012 for Cambodia). Most of the these studies were non-experimental, and thus provide less rigorous findings regarding the effects of SBM. A few recent studies of SBM effects based on experimental evidence stand out because they specifically explore the effects of supporting interventions designed to enhance the role of the school council in school decisionmaking (i.e. training, democractic elections). Although the body of evidence is still small, these more recent, experimental studies suggest positive effects of SBM on student outcomes. In Indonesia, a study of SBM found improvements in student achievement when a wider base of the community is represented in the school council. In 2006, researchers carried out an experimental pilot study to test the effects of three measures aimed at helping school committees expand their role in school decisionmaking: increased capacity and knowledge of the committee (through training and financial resources), increased community representation in the committee (through democratic elections), and improved ties between the committee and a local (village) governing body (Pradhan et al., 2011). The study's findings suggest that interventions that reinforce existing school committee structures (grant and training) have limited impacts on learning. However, interventions that change these structures, through democratic elections that allow a wider base of the community to be represented in the school committee, or by linking committees to the larger community (village), were associated with increased test scores in Indonesian language between 0.17 and 0.22 of a standard deviation (SD). In Mexico, the Apoyo a la Gestión Escolar (AGE) program was first introduced in the 1996/97 school year. AGE is designed to financially support the work of schools’ parent associations. Monetary support varies from US$500 to US$700 per year depending on school size and the use of funds goes mostly to infrastructure improvements and small civil works. In return for the grant, parents must commit to greater involvement in school activities, undertaking of the infrastructure improvements and civil works and attending training sessions delivered by state education authorities ( Bruns, Filmer, & Patrinos, 2011). A recent randomized impact evaluation of the program found that AGE schools had reduced their dropout rate by1.5-1.7 percentage points compared to control schools, during the first year of implementation. As part of the experiment, a pilot program that doubled the AGE cash grant was also tested. This pilot had a positive effect on Spanish and mathematics test scores of around 5 percent and 6-8 percent, respectively. This effect was observed mostly for third grade students ( Gertler, Patrinos & Rodriguez-Oreggia, 2010). Mexican education authorities followed-up AGE with the Programa Escuelas de Calidad (PEC) program in 2001/02. PEC was initially targeted to urban areas, and was one of the earliest large-scale SBM reforms in a developing country. PEC provides cash grants averaging around US $4,500 dollars to schools in exchange for collaboration between principals, teachers and parents for school planning and decision making. 1 The program provides some training to school principals, but not to school councils. Evaluations of the program using quasi-experimental methodologies have found small, but significant positive effects of PEC on indicators of school quality such as dropout, repetition, and failure rates ( Skoufias and Shapiro, 2006, Shapiro and Skoufias, 2006 and Murnane et al., 2006). None of the previously cited studies of PEC used test scores as an outcome variable. This paper adds to the evidence base on the effects of SBM reforms on student achievement through an evaluation of the “Programa de Fortalecimiento e Inversión Directa a las Escuelas” or “Program to Strengthen and Invest Directly in Schools” (PEC-FIDE) in Mexico. Since 2006/2007, Mexico has a national standardized evaluation “ENLACE” in math and Spanish. All students from third to ninth grade in both public and private schools are required to take this test. We use ENLACE test scores as the measure of student achievement. The program was launched in 2008-09 as a spin-off of PEC and implemented in six Mexican states: Hidalgo, Guanajuato, Estado de México, Quintana Roo, Chihuahua, and Coahuila. PEC-FIDE is financed with matching grants from states and the federal government. A state program coordinator manages PEC-FIDE's operations in each state. In the year this study took place, over 220 primary schools in the six states participated in the program. Schools that have participated in PEC are eligible for PEC-FIDE, but a school cannot receive funds from both programs simultaneously. Like PEC, PEC-FIDE seeks to improve student achievement and reduce dropout rates by providing cash grants to schools in exchange for collaborative school planning and shared decision making. The amount of PEC-FIDE's cash grant depends on school enrollment, with the average school grant being around US $4,500, at the average U.S. dollar to Mexican Peso exchange rate of 2008, and larger schools receiving upwards of US $20,000 dollars per year. Program regulations specify a suggested allocation of funds for allowable expenses: training, interventions for at-risk students, materials or equipment, and infrastructure improvements. Under PEC-FIDE, school councils--a crucial aspect of most SBM reforms-- include the principal, teacher representatives, and parent representatives. School councils oversee both school planning activities and their implementation, drafting a five-year School Improvement Plan and a one-year work plan. Many schools also have parent associations that may or may not be formally involved in the school council. By law, parental associations exist in all schools in Mexico, but they can be dysfunctional or have little access to schools (Gertler, Patrinos, and Rubio Codina, 2012). Parent associations are usually tasked with fundraising activities and providing volunteer labor for school maintenance and construction. School principals and the president of the school council receive basic training on SBM and planning offered by PEC-FIDE. In some instances the school council's treasurer participates in this training as well. Neither schools nor school zones self-select into the program, but are rather assigned to it by State program coordinators based on program targeting guidelines. These guidelines instruct state coordinators to target school zones in high poverty areas and those with low test scores in ENLACE. After school zones are selected, all schools within that zone participate in the program. However, data from our study revealed that program targeting did not follow the guidelines consistently. We found PEC-FIDE schools in low poverty areas and/or with ENLACE test scores above the average.2 In this paper, we present research on the effects of PEC-FIDE on student achievement and other outcomes. We use data from the program, the Mexican school census and ENLACE test scores, to estimate the effect of participation on student test scores, pass and dropout rates using a differences-in-differences approach. We take advantage of pre-program data to test the parallel trends assumption that supports this strategy. We test for robustness of our findings to alternate specifications using a propensity-score matching methodology. Lastly, to better understand the mechanism that would explain any observed effect (or lack thereof), we present results from a qualitative analysis of program implementation. As part of this analysis, we gathered primary data from school observations and over 200 interviews with program officials, school zone inspectors, principals, teachers, and parents. Unlike prior published research on SBM in Mexico, this study utilizes test score data which allows us to investigate effects on student learning. In addition, the availability of pre-program data allows us to control for baseline learning indicators, as well as rule out the existence of differential pre-program trends between treated and untreated schools in student test scores and other outcomes. Furthermore, our study expands on prior knowledge of SBM programs in Mexico and around the world by complementing the regression analysis with program implementation findings. Our results suggest that PEC-FIDE significantly improves Spanish test scores for 3rd graders by about 28 points in ENLACE. The magnitude of these results is substantial, close to half a standard deviation. This is equivalent to moving a student from the 50th to the 68th percentile in the spanish test score distribution for all schools in the sample. Another way to interpret the magnitude of the results is to compare the test score increase to the overall achievement gap in Spanish between public and private schools in Mexico, which is about 105 points.3 The effect of the program would correspond to about 25% of said achievement gap. If the effects of the program were to be sustained over time, public schools in our sample would close the gap with private schools in four years. The program does not appear to have had significant positive effects on any of the other outcomes under analysis. The positive effect of PEC-FIDE on Spanish test scores was only observed in schools that had not participated in PEC before-- about 40% of the sample of PEC-FIDE schools fell in this category. This result is somewhat counterintuitive given the emphasis of SBM reforms on changing governance structures and promoting greater parental involvement. Fostering a school governance structure that enables higher accountability and better use of resources is one of SBM's key tenets. Under SBM, the cash grant serves as the catalyzer to yield such change. In theory, added financial resources coupled with a more inclusive governance structure and greater parental involvement and monitoring of school activities should lead to more efficient spending and improved student outcomes. If the logic of SBM programs bears out, PEC-FIDE schools that were part of PEC in the past should have built local capacity for shared planning and decisionmaking to enable more efficient spending of the PEC-FIDE grant. Thus, we would have expected that schools that participated in PEC before produce better student learning outcomes than schools that were participating in SBM for the first time. The fact that this was not observed suggests that SBM's positive results are likely to be driven by the immediate benefits of increased school-level spending and not necessarily by producing better or more inclusive governance structures. Qualitative findings of PEC-FIDE implementation support this conclusion. The implementation analysis did not find strong evidence to suggest that school councils engaged in collaborative planning or shared decision making. In most cases, planning was done entirely by the principal with no consultation of teachers or parents. The role of the School Council was mostly limited to rubber stamping decisions already made by the school principal. This limited role of school councils has been observed in other SBM programs. A detailed implementation study of Indonesia's Bantuan Operasional Sekolah (BOS) program found that most school committees rarely met, and were seldom involved in school decision making processes such as creating a mission statement, allocating BOS funds, and developing the annual improvement plan. The school committee chair was often simply asked to sign off on decisions that had already been made by the principal (Vernez, Marshall, & Karam, 2012). Building the capacity of school-level actors for meaningful school planning and involvement is an important aspect of SBM. Our analysis suggests that one key effect of school council involvement is to monitor that resources reach the schools. Other aspects of this capacity building effort, such as effective school planning, using resources where they would matter most, and involving a broader range of actors in school decision making, might need additional program intervention. Our implementation findings suggest that despite the motivation that comes from the cash grant, school councils are not able to build this capacity by themselves. Taken together, the quantitative and qualitative results in this paper suggest that the program's positive effects are likely to be the result of giving schools financial resources to meet pressing equipment, materials, and infrastructure needs. It should be underscored that the fact that these resources were able to reach the classroom was likely the result of greater oversight by the school council. In this respect, SBM is working to effect positive governance change. SBM reforms do appear to benefit schools in Mexico, as well as schools in other developing countries. However, additional investments in training and capacity building at the school level might be needed if these reforms expect to promote structural school governance changes that can make school funding more effective.