Contemporary leadership theory and research has largely abandoned earlier approaches, such as the search for one best leadership style, and may be characterized as currently focusing on situational moderators of leadership effects (Bass, 1990 and Zaccaro and Klimoski, 2001). Unfortunately, however, “the context in which leadership is enacted has not received much attention” as a critical factor in leadership research (Antonakis, Schriesheim, Donovan, Gopalakrishna-Pillai, Pellegrini, & Rossomme, 2004, p. 48).
Due to this level of relative inattention, a number of scholars have called for the increased incorporation of context into organizational behavior research in general (e.g., Johns, 2001 and Rousseau and Fried, 2001) and into leadership research in particular (e.g., Lowe & Gardner, 2000). Context may constrain the variability that can be measured (Rousseau & Fried, 2001) and it may serve as a boundary condition for many leadership phenomena (House & Aditya, 1997). As a result, leadership processes may be incorrectly modeled and incorrect conclusions drawn if the leadership context is not properly considered (Antonakis et al., 2003 and Antonakis et al., 2004). Although many potential contextual factors apparently exist (cf. Antonakis et al., 2003 and Antonakis et al., 2004), the hierarchical level at which leadership is exercised may be an important contextual consideration in the nature of leadership and in its impact on different outcome variables. In particular, “the level at which leadership operates may vary from individual to group, or organizational level, depending on the context (e.g., CEO-level vs. supervisory-level leadership) in which it is enacted” (Antonakis et al., 2004, p. 62).
One advance in the field that has helped pave the way for a more comprehensive consideration of leadership context is the development of meso or multilevel and cross level theory and research (House, Rousseau, & Thomas-Hunt, 1995). Organizations are inherently multilevel because individuals work in groups, departments, organizations, industries, and other forms of “units” (Klein et al., 1994 and Rousseau, 1985) and “findings at one level of analysis do not generalize neatly and exactly to other levels of analysis, except under very restrictive circumstances” (Klein & Kozlowski, 2000, p. 213). Thus, the level of analysis at which a variable or process operates may be considered a form of contextual effect (a boundary condition) as it may limit variability or constrain a particular variable or effect altogether at one or more levels of analysis (Antonakis et al., 2004). Thus, leadership is clearly a research domain that warrants examination of level of analysis, since leadership manifests at multiple organizational levels.
Levels of analysis issues have increasingly become a central focus of a significant volume of recent leadership research (e.g., Avolio and Yammarino, 2002 and Yammarino and Dansereau, 2002). This reflects increased recognition of the importance of ensuring alignment between theory and the data that are used to test it (Klein & Kozlowski, 2000). Without proper alignment of theory and data, a theory cannot be subjected to valid testing nor can valid conclusions be drawn from improperly aligned data or analyses (Dansereau et al., 1984, House et al., 1995, Klein et al., 1994 and Rousseau, 1985).
Klein & Kozlowski (2000, p. 233) argue that “multilevel research… should begin with well-developed theory, and careful identification of key constructs. Construct definition includes the specification and theoretical justification of the level and nature of the construct.” Two recent reviews have presented extensive theoretical discussions and illustrations of why care must be exercised to align theory and the data and analyses that are used to test it (Yammarino et al., 2002 and Yammarino et al., 2005). These two reviews have also extensively documented that the majority of leadership research is seriously deficient with respect to addressing fundamental levels of analysis concerns (Yammarino et al., 2002 and Yammarino et al., 2005). In particular, the levels at which phenomena are expected to operate or the levels at which relationships are expected to hold are frequently not specified in the theory that is developed and tested. Additionally, the levels of analysis for which certain measures are appropriate and the use of certain analytic practices (such as aggregating individual scores to obtain dyad or group scores) are often not theoretically justified or even mentioned (Yammarino et al., 2002 and Yammarino et al., 2005).
Ostroff (1993) notes that relationships that hold at one level of analysis may be stronger or weaker at a different level of analysis, or may even reverse their direction. Klein & Kozlowski (2000) note that, “ecological correlations based on aggregate data are generally inflated estimates of lower level relationships” (p. 213) and that relationships found at lower levels may not hold at higher levels. Antonakis et al. (2004, pp. 63–66) present four illustrative examples of research where incorrect conclusions would have been likely if level of analysis had not been specifically addressed. In one example, a data set analyzed at the individual level of analysis did not detect the moderating effect of leadership climate on follower-perceived task significance because the effect did not operate at the individual level of analysis (Bliese, Halverson, & Schriesheim, 2002). However, when tested using multilevel methods, the existence of a group-level moderator effect was apparent (Bliese and Halverson, 2002, Gavin and Hofmann, 2002 and Markham and Halverson, 2002). In a second example, Yammarino (1990) found that the correlation between a leader's group-directed initiating structure behavior and follower role ambiguity perceptions was significant at the individual level of analysis (but not at the group level). However, the correlation between group-directed consideration and role ambiguity was significant at the group level of analysis (and not at the individual level).
Although there are a number of areas within the leadership domain where there is widespread and apparently strong research interest, probably the currently most studied area is that of transformational and transactional leadership, most often measured by the recent research version (Form 5X) of the Multifactor Leadership Questionnaire (MLQ5X; Avolio & Bass, 2004). The MLQ5X is regarded as “the most popular instrument for measuring transformational leadership” (Antonakis & House, 2002, p. 18). In fact, recent reviews have commented on both the explosion of activity in this area and on the fact that use of the MLQ appears to dominate research on transformational and transactional leadership (Dumdum et al., 2002 and Judge and Piccolo, 2004).
There have been a number of studies and critiques of the psychometric properties of the MLQ (e.g., Antonakis et al., 2003, Tejeda et al., 2001 and Tepper and Percy, 1994). Unfortunately, however, one shortcoming of these prior reviews and the MLQ that has not been recognized or addressed to-date has to do with its level of analysis. Simply put, while the level of analysis of some MLQ items is clear, the majority of items do not have a clear referent and, consequently, their use may lead to invalid results and problems with interpretation because the actual level of the data (and analyses) may not align with or match the theory under investigation.
Briefly stated, the purpose of this paper is to examine the theoretical level of analysis of each MLQ5X item, to determine whether there is, in fact, a basis for concern about this psychometric property of the MLQ5X. Below, we provide theory and argument for why we believe that the theoretical level of analysis at which the MLQ5X measures its constructs is problematic. We then present the results of two content validity assessments (Schriesheim et al., 1993 and Schriesheim et al., 1999) that empirically address this concern. Finally, we conclude with a discussion of the theoretical levels of analysis at which each MLQ5X dimension appears most appropriately assessed, along with suggestions for future research and illustrative items that show how new measures of the MLQ dimensions might be created for use at multiple levels of analysis. We focus on the MLQ5X because it is the most current MLQ version and because it can be obtained free of charge from the publisher for research purposes. However, our findings and conclusions should still apply to the many earlier versions of the MLQ since they share many of the MLQ5X items.
