The paper extends the current literature on peer review journal evaluations by providing a number of insights based on the diversity of Production and Operations Management (POM) research. We provide peer review evaluations for POM research outlets, based on a sampling frame that includes a large number of POM researchers worldwide. More specifically, the paper develops and tests various hypotheses as to whether the perceived quality and relevance of a journal is affected by such factors as: (i) nature of research work (empiricists versus modelers), (ii) society membership, (iii) research productivity, (iv) geographical location, and (v) seniority. Our findings suggest that caution must be exercised when utilizing existing POM journal rankings, as some factors, particularly the difference between empiricists and modelers, may influence journal evaluation. These must be considered when addressing issues such as faculty promotions, tenure, and salary.
The end of last millennium brought about the establishment of Production and Operations Management (POM) as a key discipline in most business schools. Along with it came various efforts to evaluate the overall quality and/or relevance of various POM related publications. Given the broadly defined nature of POM related research, such efforts have focused on providing journal rankings that are based primarily on either peer perceptions (Saladin, 1985, Barman et al., 1991, Barman et al., 2001 and Soteriou et al., 1999), citation analysis (Goh et al., 1996, Goh et al., 1997 and Vokurka, 1996) or some combination (Vastag and Montabon, 2002 and Gorman and Kanet, 2005). Regardless of the methodology followed, such efforts can be of aid to various stakeholders when making decisions on issues such as faculty promotion, tenure, salary, etc. As such, they eventually shape the future of research efforts which are, in turn, communicated to the research and practitioner communities.
Despite these efforts, today no universally accepted rank order for POM journals exists. Each approach providing journal rankings has various limitations (Vastag and Montabon, 2002) and despite the best approach debate in the literature (Mitchell, 1996) most share similar weaknesses. In our view, the most important of these weaknesses is that existing rankings do not take into consideration the diversity of POM research. In the case of perceived rankings, for example, different evaluations are expected to be obtained by an empiricist, compared to those obtained by someone whose research focuses on modeling. The nature of someone's research may introduce bias when evaluating a journal. Such bias has not been addressed in previous studies. Other sources of potential diversity such as geographic location, and other respondent characteristics such as seniority, individual productivity and professional membership, may introduce further bias in perceived rankings. Such biases must be identified and subsequently considered by stakeholders during their decision making process.
Overall, this paper explores the heterogeneity present within the POM global community with respect to perceived quality and relevance of journals. The study makes three contributions that extend previous work on POM journal evaluations. First, we demonstrate that differences exist among the way researchers in the POM field evaluate journals. Potential biases resulting from philosophical and methodological differences in conducting POM research are explored and discussed. Differences regarding the way such subgroups – for example, empiricists and modelers – evaluate the quality and relevance of journals are reported. Journal evaluation differences resulting from professional society membership, and geographic location are also examined. Such differences must be taken into consideration, especially since journal rankings increasingly drive tenure and publication decisions. We also provide insights on journal evaluations based on the research productivity of respondents and examine whether researchers that publish in a specific journal also provide higher ratings for that journal. Clusters of journals are provided that clearly demonstrate the above. Third, our study provides POM journal perceived rankings by considering feedback from a sample drawn from a large sample of researchers located in North America, Europe, and Asia. With very few exceptions (Soteriou et al., 1999), most existing studies focus only on researchers from the US, although “… many of the greatest contributors to the field of POM hail from outside of the US.” (Young et al., 1996, p. 45).
The organization of the paper is as follows. Section 2 includes a brief literature review while Section 3 presents the hypotheses development. Section 4 presents the research design and further describes the survey instrument and sampling method used. Section 5 describes the results while Section 6 presents a discussion of our findings. Finally, Section 7 presents concluding remarks.
