نمونه ای در مقابل آزمون خارج از نمونه قابلیت پیش بینی بازگشتی سهام در زمینه داده کاوی

We undertake an extensive analysis of in-sample and out-of-sample tests of stock return predictability in an effort to better understand the nature of the empirical evidence on return predictability. We find that a number of financial variables appearing in the literature display both in-sample and out-of-sample predictive ability with respect to stock returns in annual data covering most of the twentieth century. In contrast to the extant literature, we demonstrate that there is little discrepancy between in-sample and out-of-sample test results once we employ out-of-sample tests with good power. While conventional wisdom holds that out-of-sample tests help guard against data mining, Inoue and Kilian [Inoue, A., Kilian, L., 2004. In-sample or out-of-sample tests of predictability: which one should we use? Econometric Reviews 23, 371–402.] recently argue that in-sample and out-of-sample tests are equally susceptible to data mining biases. Using a bootstrap procedure that explicitly accounts for data mining, we still find that certain financial variables display significant in-sample and out-of-sample predictive ability with respect to stock returns.

There now exists a voluminous literature on the predictability of stock returns from past information. Interestingly, researchers have identified a large number of financial variables that appear to predict future stock returns. These include the dividend-price ratio (Rozeff, 1984, Campbell and Shiller, 1988a, Fama and French, 1988 and Hodrick, 1992), price-earnings ratio (Campbell and Shiller, 1988b and Campbell and Shiller, 1998), book-to-market ratio (Kothari and Shanken, 1997 and Pontiff and Schall, 1998), market value-to-net worth ratio or “Fed q” (Smithers and Wright, 2000 and Robertson and Wright, 2002), dividend-payout ratio (Lamont, 1998), term and default spreads on bonds (Campbell, 1987 and Fama and French, 1989), short-term interest rate (Campbell, 1987, Hodrick, 1992 and Ang and Bekaert, 2001), equity share in total new equity and debt issues (Baker and Wurgler, 2000), and consumption-wealth ratio (Lettau and Ludvigson, 2001).2 The evidence for the predictability of stock returns comes primarily from in-sample predictive regression models. While there are important econometric difficulties relating to the lack of exogenous regressors and overlapping observations in predictive regression models (Mankiw and Shapiro, 1986, Stambaugh, 1986, Stambaugh, 1999, Richardson and Stock, 1989 and Nelson and Kim, 1993), Campbell (2000, p. 1523) nevertheless concludes, “Despite these difficulties, the evidence for predictability survives at reasonable if not overwhelming levels of statistical significance. Most financial economists appear to have accepted that aggregate returns do contain an important predictable component.” While Campbell (2000) is probably correct in his assessment, as noted above, the extant literature primarily relies on in-sample tests of stock return predictability. This raises concerns of data mining, also referred to as model overfitting or data snooping. Lo and MacKinlay (1990) and Foster et al. (1997) provide theoretical analyses of data mining in the context of return predictability. It is typically believed that out-of-sample tests provide a measure of protection against data mining, as statistical models are tested using out-of-sample observations that are not used in the estimation of the statistical model itself. It is interesting to note that the relatively few studies that employ out-of-sample tests of return predictability typically obtain negative results. For example, in an effort to guard against model overfitting, Bossaerts and Hillion (1999) use different model selection criteria to choose the best forecasting model of real stock returns for a number of industrialized countries over the postwar period. Testing for out-of-sample forecasting power by regressing actual returns on the forecasts from the best models, they find that the best forecasting models for the United States fail to have significant out-of-sample forecasting power for S&P 500 excess returns at the 1-month horizon over the 1990:06–1995:05 out-of-sample period. They thus conclude that there is no external validation of the best forecasting models. Goyal and Welch (2003) also employ out-of-sample tests. They examine the predictive ability of the dividend-price ratio for CRSP value-weighted annual excess returns over the 1926–2000 period. While they find evidence of in-sample predictability, a model that includes the dividend-price ratio exhibits little out-of-sample predictive ability compared to a model of constant returns according to the Diebold and Mariano (1995) and West (1996) statistic.3 The negative results typically generated by out-of-sample tests suggest that the in-sample evidence of return predictability is spurious. The disparities between in-sample and out-of-sample test results of return predictability in the literature make an overall assessment of return predictability difficult. In this paper, we undertake an extensive analysis of both in-sample and out-of-sample tests of stock return predictability in an effort to better understand the empirical evidence on return predictability. We test whether the financial variables cited in the opening paragraph exhibit significant in-sample and out-of-sample predictive ability with respect to stock returns on the S&P 500 and CRSP equal-weighted portfolios. We follow the literature and assess in-sample predictability via the t-statistic corresponding to the slope coefficient in a predictive regression model. In order to test for out-of-sample predictability, we compare out-of-sample forecasts generated by a model of constant returns to forecasts generated by a model that utilizes a given financial variable using two recently developed test statistics. The first, due to McCracken (2004), is a variant of the Diebold and Mariano (1995) and West (1996) statistic that tests for equal predictive ability. We also use a statistic designed to test for forecast encompassing, a variant of the Harvey et al. (1998) statistic due to Clark and McCracken (2001). Importantly, Clark and McCracken, 2001 and Clark and McCracken, 2004 find the variants to be considerably more powerful than the original statistics in extensive Monte Carlo simulations. These more powerful tests may thus be better equipped to detect out-of-sample predictability than the statistics typically used in the return predictability literature. Using annual data for 1927–1999 and S&P 500 real stock returns, we find that the equity share has significant in-sample predictive ability at the 1-year horizon, the term spread has significant in-sample predictive ability at the 5-year horizon, and the price-earnings ratio and Fed q have significant in-sample predictive ability at the 10-year horizon. At least one of the McCracken (2004) and Clark and McCracken (2001) statistics also provides significant evidence of out-of-sample predictive ability for these same variables over the 1964–1999 out-of-sample period. For this data set, there is no discrepancy between the in-sample and out-of-sample test results once we use powerful out-of-sample tests. When we measure real returns based on the CRSP equal-weighted index, we identify four variables—book-to-market ratio, Fed q, default spread, and equity share—with significant in-sample predictive ability at the 1-year horizon. The dividend-price ratio and equity share display significant in-sample predictive ability at the 5-year horizon, and the short-term interest rate evinces significant in-sample predictive ability at the 10-year horizon. Some of these variables also display significant out-of-sample predictive ability. Overall, we find relatively little discrepancy between the results from in-sample and out-of-sample tests of predictability for our annual data, and we attribute this to our use of recently developed out-of-sample tests with good power properties.4 According to “conventional wisdom,” our out-of-sample evidence of stock return predictability is more reliable than our in-sample evidence, as it is less susceptible to data mining. However, Inoue and Kilian (2004) recently challenge the conventional wisdom and argue that out-of-sample tests should not be preferred to in-sample tests, as in-sample and out-of-sample tests are equally susceptible to data mining. Indeed, Inoue and Kilian (2004) show that, if appropriate critical values are used, in-sample and out-of-sample tests of predictability are equally reliable against data mining under the null hypothesis of no predictability. 5 Given the findings in Inoue and Kilian (2004), we compute appropriate critical values for all of our in-sample and out-of-sample statistics based on a data-mining bootstrap procedure. Even when we explicitly take data mining into account through the data-mining bootstrap procedure, we still identify some financial variables with significant in-sample and out-of-sample predictive ability. As a cautionary note, we emphasize that we are concerned with testing for the existence of return predictability in population. This is a conceptually and practically distinct issue from whether a practitioner in real time could have constructed a portfolio that earns extra-normal returns. A practitioner is interested in ranking models—without necessarily caring about the significance of any differences—and selecting the best forecasting model according to a profit-based metric.6 In contrast, predictability tests are designed to test for the existence of a predictive relationship in population.7 The rest of the paper is organized as follows. Section 2 presents our econometric methodology. Section 3 reports the in-sample and out-of-sample predictability test results. Section 4 summarizes our main findings.

In the present paper, we show that there is not a great deal of discrepancy between in-sample and out-of-sample tests of stock return predictability, once we use relatively powerful out-of-sample tests. The extant literature on stock return predictability thus appears to overstate the degree of disparity between in-sample and out-of-sample test results. We also test for in-sample and out-of-sample predictability in a data-mining environment suggested by Inoue and Kilian (2004). Using a bootstrap procedure that delivers data-mining robust critical values, we find that certain financial variables, such as the equity share and Fed q, have significant in-sample and out-of-sample predictive ability. Given that we obtain evidence of predictability in an econometric environment that explicitly controls for data mining—in addition to potential biases associated with regressor endogeneity and overlapping observations—our in-sample and out-of-sample test results strengthen the case for a predictable component in stock returns.23