آزمون دقیق نمونه محدود برای رگرسیون خطی با متغیر وابسته محدود

We introduce tests for finite-sample linear regressions with heteroskedastic errors. The tests are exact, i.e., they have guaranteed type I error probabilities when bounds are known on the range of the dependent variable, without any assumptions about the noise structure. We provide upper bounds on probability of type II errors, and apply the tests to empirical data.

The fundamental goal of hypothesis testing, as set by Neyman and Pearson (1930), is the minimization of both type I and type II error probabilities. To cite Neyman and Pearson (1930, p. 100): (1) we must be able to reduce the chance of rejecting a true hypothesis to as low a value as desired; (2) the test must be so devised that it will reject the hypothesis tested when it is likely to be false. In the usual model of linear regressions, how well are these goals achieved? When error terms are normally distributed and homoskedastic, the classical test1 has a type I error probability equal to the nominal level of the test. But error terms in real data almost never have a precisely normal distribution, let alone a homoskedastic one. For any given heteroskedastic noise structure, White (1980)’s robust test guarantees a type I error probability that approaches the nominal level when the sample size goes to infinity. But without restrictions on the (unknown) noise structure, and for any sample size, the probability of a type I error resulting from the use of White (1980)’s test can be as large as 1. In fact, this is a consequence of a general impossibility result due to Bahadur and Savage (1956) and Dufour (2003) that shows that no meaningful test can be constructed in which the probability of a type I error is guaranteed to be less than 1. The use of statistical tools in situations where the underlying distributional assumptions are not satisfied can have catastrophic consequences. Practitioners can be led to greatly underestimate the probability of certain outcomes, and remain unprepared to those outcomes while thinking they are safe. This is what must have happened to David Viniar, CFO of Goldman Sachs, who declared in August 2007 about the financial crisis: “We were seeing things that were 25-standard deviation moves, several days in a row” (quoted in Larsen, 2007). Since the probability of a 25-standard deviation event under the normal distribution is less than 1 over 10137, we can safely conclude that the distributional assumptions used by Viniar and his colleagues were not satisfied. In this paper our message is a positive one. We identify an important class of statistical problems where the negative conclusions from Bahadur and Savage (1956) and Dufour (2003) do not apply, and we introduce tests with guaranteed upper bounds on type I and type II errors for this class of problems. The tests are exact in the sense that they guarantee a type I error probability below the nominal level independently of the error structure.2 We also implement our these tests in practical numerical examples. The class of problems we consider is the class in which a bound on the dependent variable is known. This condition is satisfied in a large range of applications. For instance, it is warranted by the very nature of the endogenous variable (e.g., proportions, success or failure, test scores) in 43 of the 75 papers using linear models published in 2011 in the American Economic Review. It should be noted that even under the boundedness condition, the existence of exact tests was previously an open problem. Previous exact tests were derived by Schlag, 2006 and Schlag, 2008a for the mean of a random variable and for the slope of a simple linear regression. Exact tests for linear regressions under the alternative assumption that error terms have median zero are developed by Dufour and Hallin (1993), Boldin et al. (1997), Chernozhukov et al. (2009), Coudin and Dufour (2009), Dufour and Taamouti (2010). We refer to our two tests as the nonstandardized and the Bernoulli tests. We briefly summarize their constructions here. Each test relies on a linear combination of the dependent variables (such as in the OLS method) that is an unbiased estimator of the coefficient to be tested. The nonstandardized test relies on inequalities due to Cantelli (1910), Hoeffding (1963), and Bhattacharyya (1987), as well as on the Berry–Esseen inequality (Berry, 1941, Esseen, 1942 and Shevtsova, 2010), to bound the tail probabilities of the unbiased estimator. One challenge in the construction is to apply the Berry–Esseen inequality even though there is no lower bound on the variance of any of the error terms. The Bernoulli test generalizes the methodology introduced by Schlag, 2006 and Schlag, 2008b for mean tests. Each term of the linear combination that constitutes the unbiased estimator is probabilistically transformed into a Bernoulli random variable. We then design a test for the mean of the family obtained using Hoeffding (1956)’s bound on the sum of independent Bernoulli random variables. This defines a randomized test, on which we then rely to construct a deterministic test. We provide bounds on the probabilities of type II error of our tests. These bounds can be used to select–depending on the sample size and the realization of the exogenous variables–which of our tests is most appropriate. We also rely on these bounds to show that these tests have enough power for practical applications. In two canonical numerical examples involving one covariate in addition to the constant, the bounds on the probability of type II errors show that the tests perform well even for small sample sizes (e.g., n=40n=40). We implement our tests and compute confidence intervals using the empirical data from Duflo et al. (2011).3 We compare the results relying on our test with the 95/ ones obtained using either the classical method and White’s heteroskedastic robust method. The results show that, compared to the classical test or White’s test, the losses of significance of our exact method are moderate, and the confidence intervals are in most cases augmented by a factor of no more than 50%. The paper is organized as follows. Section 2 introduces the model. Sections 3 and 4 present the nonstandardized test and the Bernoulli test. In Section 5, we examine their efficiency using numerical examples. Section 7 shows an application of the tests to empirical data. The underlying data-generating process is discussed and extensions are discussed in Section 8. We conclude in Section 9. All proofs are presented in the Appendix A and Appendix B.

This paper introduces finite-sample methods that are exact in the sense that they do not rely on assumptions about the noise terms beyond independence. These tests perform well even in small sample sizes (n=40,60n=40,60). They are powerful enough to allow practical conclusions to be drawn when they are applied to independently collected empirical data. The nonstandardized test relies on a selection of probabilistic bounds. Improvements of these bounds would lead to an improved test. We note, however, that when we conducted a thorough, albeit nonexhaustive, examination of bounds derived from a series of known inequalities such as those from Bercu and Touati (2008), Bernstein (1946), Pinelis (2007), and Xia (2008), these bounds did not result in any improvement of our method.10