The use of internet-based questionnaires, also known as “e-questionnaires” (Denscombe, 2006, p. 252), as a research tool has increased significantly over the past decade. Compared with their traditional pen-and-paper counterparts, e-questionnaires are relatively inexpensive, easy to administer, can easily cover wide geographical areas, and tend to have higher completion rates (Denscombe, 2006). Furthermore, this method of data collection avoids a number of the disadvantages of standard pen-and-paper administration, such as the potential for participants to fail to answer questions and the time-consuming need to transcribe data (Mangunkusumo et al., 2005). Responses given anonymously via e-questionnaires, when compared to anonymous mail-returned pen-and-paper methods, have been found to result in lower scores on social desirability measures (Joinson, 1999), and have been found to generate greater self-disclosure from participants (Stanton, 1998).
E-questionnaires are typically on-line versions of widely used, popular questionnaires that have been shown to have good psychometric properties when completed by participants via pen-and-paper methods. However, it cannot be assumed a priori that the psychometric properties of an internet-based version of tried and tested questionnaires will be identical to those of the pen-and-paper version ( Ritter, Lorig, Laurent, & Matthews, 2004). There may exist what has been termed a “mode effect”, a differential pattern of responding by participants depending on the medium of administration of the questionnaire (e.g., Denscombe, 2006 and McCabe, 2004). Richman, Kiesler, Weisband, and Drasgow (1999) have noted that certain properties of e-questionnaires such as the display of plain text on a screen, the ephemeral nature of responses, and the absence of social context cues may affect participant response patterns. However, preliminary evidence suggests that this mode effect is not significant ( Richman et al., 1999). For example, studies have compared the data obtained from questionnaires administered both as e-questionnaires and via pen-and-paper format in areas such as personality ( Buchanan & Smith, 1999), health assessment ( Ritter et al., 2004), illicit drug taking ( McCabe, 2004), and even library quality ( Hayslett & Wildemuth, 2004). Such studies have not found significant differences in participant responses.
One area in which e-questionnaires are likely to be particularly useful is in the assessment of psychopathological experiences in the healthy population. The study of such experiences has been given a renewed momentum by findings that they exist on a continuum extending into the normal population (Johns and van Os, 2001 and Peters et al., 2004). Finding reliable self-report instruments for assessing these experiences is thus likely to be important for the future development of research in this area. Given that psychopathology questionnaires are likely to be particularly sensitive to social desirability effects, it is possible that e-questionnaire versions of standard pen-and-paper psychopathology questionnaires may result in significantly different patterns of response. Investigating whether there is a mode effect for psychopathology questionnaires would therefore seem to be particularly timely given the number of studies that are starting to use this method (e.g., Allen et al., 2005, Berry et al., 2006, Jones and Fernyhough, 2006 and Jones and Fernyhough, 2007).
The present study set out to examine if a mode effect was present when standard measures of psychopathology in the healthy population were administered as e-questionnaires. Equivalence between the two modes of administration was to be assessed in two ways. Firstly, the means of the psychopathology instruments were to be compared by mode of administration. This is a popular method of establishing mode equivalence of questionnaires (e.g., Cronk & West, 2002). We hypothesized that, due to the lower social desirability scores and greater self-disclosure associated with e-questionnaires, self-reported proneness to these unusual experiences would be higher than would be expected with pen-and-paper administration.
We also set out to examine if the factor structures of the psychopathology questionnaires differed by mode of administration. If equivalent factor structures are found for both modes of administration this offers evidence that the same constructs are being measured in each mode (Buchanan & Smith, 1999). This comparison was to be performed using multi-group confirmatory factor analysis. We hypothesized that the factor structures of the psychopathology questionnaires employed here would be mode invariant. Finally, it was hypothesized that the e-questionnaires would be as satisfactory in their internal reliability as their pen-and-paper counterparts.
