تعصبات متفاوت در برآورد تطابق: شواهد حاصل از دو آزمایش تصادفی آموزش جستجوی کار

We compare various matching estimators with the results from two randomised field experiments that evaluate the employment effects of job search training programmes. We find that commonly used non-experimental matching estimators tend to overestimate the programme effects, especially in the first experiment in which participation in the programme is voluntary. In the second experiment, where caseworkers assign unemployed persons to the training programme, the matching methods produce estimates that are close to the experimental results.

In 1996 the Finnish Institute of Occupational Health carried out a demonstration project “Työhön”, which evaluated the employment effects of a job search training programme, using a randomised field experiment. The participants were selected by randomly allocating half of the applicants to a training course and the other half to a control group that did not participate in training. Three years after the first experiment, job search training programmes were introduced nationwide in an attempt to increase active elements within labour market policies. In connection with this reform, another randomised field experiment was carried out in nineteen employment offices across the country. Again, participants were randomly chosen, this time by assigning two-thirds of the eligible applicants to a job search training course and leaving one-third in the control group. In both cases over one thousand unemployed persons took part in the experiment. The demonstration project closely followed the intervention design with the researchers monitoring its implementation. The latter courses were the usual job search training courses that applied various training methods and that were organized by local employment offices. One essential difference involved the selection process. Participation in the demonstration project was entirely voluntary and the participants were recruited by means of advertisements at local employment offices and in local newspapers. In the second experiment the participants in the programme were first selected by the employment offices using their usual procedures and, after having already been assigned to a programme, they were then asked whether they were willing to take part in a randomised experiment. The employment effects of these job search training experiments have been documented previously by Vuori et al. (2002) and Malmberg-Heimonen and Vuori (2005). These studies used data from the follow-up surveys conducted six months after programme participation. Vuori and Silvonen (2005) also report results based on a second follow-up study two years after the experiment. None of these studies found significant main effects, but Vuori and Silvonen (2005) report positive employment effects among participants who are at risk of depression. Several previous studies have analysed the effects of job search training but in most cases job search training is part of a larger programme package. For example, Hotz et al. (2006) re-evaluate experimental results from the California GAIN programme. Dolton and O'Neill (2002) evaluate the Restart programme, and Blundell et al. (2004) the New Deal programme in the UK. Both programmes include not only job search training but also other elements such as monitoring, basic skills courses or subsidized job placement. Van den Berg and Van der Klaauw (2006) present results from a randomised social experiment that evaluates the effects of job search assistance and tighter monitoring in the Netherlands. With the exception of the Dutch study, the results show that job search assistance, possibly combined with other elements, improves the participants' employment prospects. In this paper we evaluate the long-term effects of Finnish job search training experiments, using administrative register data on the participants that allows us to trace the effects of job search training on a monthly basis from the date of randomisation up to six years after participation. We then estimate the bias that would result if the experimental set-up were not available and the programme effects would be estimated using standard non-experimental matching methods. Our paper contributes to the literature that compares non-experimental estimates with the experimental results, a tradition started by the LaLonde (1986) analysis of the National Supported Work Programme. Much of the more recent research has focused on another large-scale randomised experiment evaluating the effects of the Job Training Partnership Act described in detail by Bloom et al. (1997). A thorough analysis that compares the bias of several non-experimental estimators is presented by Heckman et al. (1998). Their conclusion is that bias in the non-experimental estimates can be substantially reduced by using data on non-participants from the same labour markets, administering the same survey instrument to both groups and by including information on recent labour market histories. However, so far, experimental evidence on the effects of labour market programmes comes mainly from the U.S. The ability of non-experimental methods to replicate experimental results depends on the available data and on the selection process involved in the programmes. These processes vary across countries and across different programmes within individual countries. In this study we will be able to evaluate how different selection processes influence the bias in the non-experimental procedures by comparing the results based on two different experiments on job search training that differ in how participants are selected for the programme. The rest of this paper is organized as follows. Section 2 describes the intervention and the design of randomised field experiments. Section 3 discusses the non-experimental methods. In Section 4 we report how the participants in the experiment were traced from the administrative registers and we compare the survey responses to the register data. In Section 5 we report the long-term outcomes of the job search training programmes and compare these results with non-experimental matching estimators. Section 6 concludes.

In this paper we used data from two different randomised experiments with different selection processes for participants. The results are somewhat different. Typical matching methods yield reasonably good estimates when the selection for the programme is based on the caseworker's assessment. In contrast, even a large number of covariates do not get rid of the selectivity bias when programme participation is entirely voluntary and perhaps largely determined by the applicants' unobserved motivational or other characteristics. In this case, the resulting bias is substantial compared with the experimental estimate and could lead to quite different policy conclusions. Since selection processes differ across programmes, results showing that non-experimental methods “work well” in a particular setting cannot be easily generalized to cover other programmes.