Since the early 1970s, there has been an increasing interest in research about people's understanding and performance in probability and statistics. Researchers especially focused on misconceptions and heuristics regarding probability, chance, and randomness (e.g., Kahneman, Slovic, & Tversky, 1982; Konold, 1989 and Konold, 1991; Nisbett & Ross, 1980; Shaughnessy, 2003). This article reports on a systematic review of the available empirical evidence of students’ misconceptions in statistical inference. It constitutes a starting point for educational researchers interested in the relation between misconceptions and the conceptual change theory (e.g., Finch & Cumming, 1998; Guzzetti, Snyder, Glass, & Gamas, 1993; Smith, diSessa, & Roschelle, 1993), or a helpful tool for teachers of statistical inference to become aware of the most common misconceptions that their students may hold (e.g., Batanero, Godino, Vallecillos, & Holmes, 1994; Brewer, 1985; Haller & Krauss, 2002).
In educational research, the term misconception is used to refer to several concepts. On the one hand, authors often consider a broad definition of the word, using it to label different concepts such as preconception, misunderstanding, misuse, or misinterpretation interchangeably ( Smith et al., 1993). On the other hand, misconceptions are sometimes defined in a more restrictive way, as misunderstandings generated during instruction, emphasizing a distinction with alternative conceptions resulting from ordinary life and experience ( Guzzetti et al., 1993). In this manuscript, a refinement of the first definition is applied, and the term refers to any sort of fallacies, misunderstandings, misuses, or misinterpretations of concepts, provided that they result in a documented systematic pattern of error ( Cohen, Smith, Chechile, Burns, & Tsai, 1996).
The interest on statistical inference arises from three realities. First, this is a topic of main relevance for the development of research in all empirical sciences in general and psychology and education in particular ( Belia, Fidler, Williams, & Cumming, 2005; Krauss & Wassner, 2002). Second, inference receives special attention in statistical courses from almost all scientific areas, where hypotheses tests and confidence intervals are taught to students as the methods for evaluating scientific hypotheses ( Aberson, Berger, Healy, & Romero, 2003; APA, 2001). Finally, inferential ideas seem to be especially sensitive to be misunderstood and students are often prone to fall into deep misconceptions ( Daniel, 1998 and Kirk, 2001) because they require students to understand and connect many abstract concepts such as sampling distribution and significance level.
After presenting our methodology of search for this review (Section 2), we provide an overview of the misconceptions mentioned and exemplified in the literature, and describe to what extent and under which conditions they occur, discussing the methodology of the presented group of studies (Section 3). Finally, we conclude with some suggestions for further research (Section 4).
As our findings show, the literature on statistics education, and particularly publications providing empirical evidence of misconceptions in statistics, is sparse (see also Batanero, 2005; Ware & Chastain, 1991). The four searches defined above yielded more than 500 references that contained only 21 publications (cf. Table 1) reporting on 17 different studies (cf. Table A1) that provide evidence of misconceptions about topics related to statistical inference and that satisfied our selection criteria. Whereas some of the misconceptions were described several times, for others we found none or only one or two empirical studies.
