درک نقش پیش بینی ساز در پیش بینی حد برآورد اثرات سیاست: در مورد سیاست های مدیریت تقاضای سفر

Forecasting the impacts of a proposed policy is an important component of the transportation planning and decision making process. Although scientific tools are often used in transportation forecasts, biases and, more specifically, overestimations of the expected impact are often observed. This study explores the correlations between forecast-maker’s characteristics and forecast bias creation and reduction. The study examines two transport-related policies aiming at the reduction of car use: telecommuting and carsharing. Both are Travel Demand Management (TDM) policies, which attract much attention from transport experts. We tested the extent to which the forecast-maker’s beliefs about the policy at stake affected the forecast bias. We found that attitudes and beliefs associates not only with overestimation bias but also with its reduction over time. We also tested the extent to which the forecast-maker’s affiliation, the performing institute and the publication type were correlated with the biases of the forecast and with the forecaster attitudes and beliefs. These characteristics are intuitively used by the forecast user as tools to assess the ‘objectivity’ of the forecast, but our analysis found no association between these characteristics and the forecast bias.

Forecasting policy impacts play a significant role in the rational decision making process. Such forecasts lay the basis for ex-post policy evaluations and are perceived as an important scientific component of the policy rationale. However, although rigorous tools are used to predict policy impacts in many of the forecasts, biases in forecasts are well-known and empirical studies have found systematic biases associated with types of predictors and their clients (for example, a comparison between economic forecasts made by different actors and administrations was reported by Frendreis and Tatalovich, 2000 and Baghestani, 2008 compared unemployment rate predictions; Boylan (2008) found that the budget in a year ending right before an election was based on overly optimistic forecasts). In fact, optimistic factors have also been identified in the private sector concerning a systematic overestimation of the success of projects and investments (Lovallo and Kahneman, 2003). The literature suggests three possible explanations for the underlying causes of such overly-optimistic predictions regarding the impacts of transport policies. The first claims that these overestimations are not biases, but simply errors which stem from the inherent uncertainty in the forecasting of future human behavior (Mierzejewski, 1996, Niles and Nelson, 2001, Richmond, 1990 and Rodier et al., 2001). However, if this were true, we should observe both overestimation and underestimation forecasts, when in fact we see mostly overestimation (and hardly ever underestimation). The second explanation attributes overestimation to optimism bias, in which one’s belief that a project or policy one is engaged with will be successful affects, consciously or unconsciously, the study results (Flyvbjerg, 2008). The third explanation attributes overestimation to deliberate bias, also known as strategic misrepresentation, in which one purposefully skews results to serve institutional and private needs. In many cases, overestimation results from a combination of optimism bias and deliberate bias (Mackie and Preston, 1998 and Wachs, 1989). A similar explanation has been suggested by Laster et al. (1999) to rationalize the bias in economic forecast performance by private firms. In this case, the motivation for publicity overpowers the motivation for accuracy and results in biased forecasts. Although these explanations implicitly suggest that overestimation is associated with the type and the motivation of the forecasters, the role of the attitudes and beliefs of the people producing the forecasts on overestimation bias remain unclear. The existing research on biased forecasts for the impact of transportation policies focuses on two major issues: mega-projects and public transportation (see for example: Meyer, 1999, Flyvbjerg et al., 2002, Flyvbjerg, 2008, Mackett and Edwards, 1998, Pickrell, 1990 and Wachs, 1990). All biases have the same direction: overestimation of the benefit of transportation investments. Although briefly mentioned in a few studies (Howe, 1980, Brinkman, 2003, Wachs, 1990 and Wachs, 2001), overestimation in transportation policy has not been examined systematically as a factor of the people who are preparing the forecasts regardless of their organization affiliation. This paper focuses on the professionals who make the forecasts (forecast-makers) and asks whether we can identify certain types of forecast-makers that are likely to produce over-optimistic forecasts, while other are likely to make less biased predictions. As we show later, over time the bias in predictions is reduced when different forecast-makers enter the scientific discourse. Therefore, it is interesting to examine various characteristics of the forecast-makers and test what are the characteristics that may explain the differences in forecasting. We examine two types of forecast-maker’ characteristics. First, we test the impact of the forecast-maker norms and beliefs on the forecast results. We refer these norms and beliefs as “unrevealed characteristics” as these are hidden characteristics. The second type of forecast-makers’ characteristics are referred as “revealed characteristics” since we can observe them easily. These include author affiliation, performing institute and publication type (scientific publication versus other types of publications). Forecast users (whether policymakers, scientists and others) usually tend to evaluate the quality and the objectivity of a research or forecast according to three determinants: salience, legitimacy, and credibility (Clark et al., 2006). These ‘revealed indicators’ are oftentimes the only indicators available to the readers. We test the common belief that a scientific product categorized as less biased, and explores potential gaps between the revealed characteristics of the forecast-makers and their norms and beliefs. Furthermore, this paper is the first to examine not only the factors that may be associated with creating the overestimation but also how forecast-makers who act to reduce overestimation bias affect the field. We examine the correlation of the revealed and unrevealed forecast-makers indicators associated with overestimation of Travel Demand Management (TDM) impacts. TDM policies are transport-related policies aiming at reducing car travel. The studies of two TDM policies are examined: telecommuting and carsharing. Carsharing is a short-term car rental service which was first introduced in North America in the 1990s. Telecommuting, a much older policy, first suggested in the 1960s (NAE, 1969), is based on substituting commute trips with working from home using electronic communication, and refers to employees who can work from their conventional work place but instead work from home (Giuliano, 1992). These policies aim to change travel behavior and reduce the use of private vehicles by offering other mode of access. Therefore, forecasts and ex-post evaluations of the impact of these policies are concerned with the ability of telecommuting and carsharing to reduce car mileages. These predictions are at the heart of this study.

Classification according to optimistic and skeptical policy core beliefs may explain reduction in overestimation over time, as occurred in telecommuting, and may allow us to predict delay in reduction in the absence of skeptics in the process. Existing theories that focus on optimism bias and intentional bias can provide only a partial explanation of the reduction in forecasts over time. The results of this study suggest that experts are not homogeneous in their beliefs, and therefore, in the biases they introduce into their work. The introduction of skeptics, as a balance to optimists, suggests an explanation for such a reduction, even though it is a slow process. It appears that the role of scientific skepticism is not only to point out the optimism bias but also to fill the gaps in data and knowledge. Research on the role of the beliefs of forecast-makers and researchers in general, and skeptics specifically, can help to better understand and reduce the impact of optimism bias and strategic misrepresentation that in many cases is unavoidable with new policies. Furthermore for a policy maker, expert, journalist or layperson who reads a study or report in a field he is not familiar with, the affiliation of the author and the institute preparing the study are probably the only way to assess its objectivity. Similarly, even scientists tend to give higher credibility to scientific papers over other formats of publication. The results of this research imply that this classification may not be a valid indicator of potential bias in many cases because of biases that stem from beliefs that are not correlated with affiliation. This paper calls for a new way of classifying experts’ knowledge in studies and for a consequent change in the use of this knowledge in the policy making process. It is also call for policies that will introduce studies lead by scientific skeptics in early phases of the policy introduction in order to reduce overestimation bias.