رنکینگ، رتبه بندی و نمره دهی مجلات توریسم: چالش های میان رشته ای و نوآوری

This paper provides a critical analysis of journal ranking and citation analysis in tourism studies. Authors and institutions using journal impact factors, citation frequency and hits as measures of academic productivity or importance should exercise great care in their use. A close look at Journal Citation Reports produced by Social Science Citation Index, ‘hits’ on CAB International, and citation analyses from several databases shows that the desire for a universal ranking system has so far only generated some imperfect systems and inconsistent applications to suit different needs. One size simply does not fit all. Drawing upon insights from other fields that have been addressing similar ranking and citation issues, concrete suggestions are offered for developing alternative evaluation parameters and processes for managing the diverse range of interdisciplinary journals in tourism and hospitality. Specifically, the analysis argues for differentiating journals by scope, influence, relevance and quality, and a scoring system that involves participation from the community of social scientists. Innovations for timely, effective dissemination of tourism knowledge are also forwarded.

First, rankings create incentives for journal editors to select certain types of manuscripts. Second, the journal editors’ desires to select certain types of manuscripts can create incentives for authors seeking publication in those journals to produce those types of manuscripts. (Korobkin (1999, p. 853). Florida State University Law Review) Referring to the endeavor of ranking law journals (above), Korobkin (1999) says the important issue is “how rankings can be devised to encourage the future production of valuable scholarship.” We address the same problem here. Time and time again, those working and studying in post-graduate academic programs engaged in tourism studies are faced with perplexing decisions related to manuscript publishing and journal acquisition. The proliferation of new tourism journals with diverse orientations, the recognition and hiring of disciplinary scholars within interdisciplinary concentrations, plus growing concern and debate over journal ranking, rating and relevance, all point to a field that is struggling with epistemological and legitimation issues.1 A number of challenges revolve around tenure and promotion decisions, library acquisitions and securing academic positions. For Ph.D. students and new scholars seeking to disseminate their research in legitimate, widely read outlets, the long wait from manuscript submission to publication in the top three tourism journals, Annals of Tourism Research, Tourism Management and Journal of Travel Research, is a worry. 2 Selecting journals for publication and acquisition raises some common issues for those impacted by this activity: Which journal to select in the burgeoning new journal domain? How is it viewed relative to the top three tourism journals? What exactly makes these three ‘top tier’ journals in tourism studies? What about new online journals? How do the ranking/rating schemes we use deal with the interdisciplinary nature of tourism research? What constitutes scholarship or academic leadership in tourism studies and how is this evaluated? A number of recent articles have offered research and recommendations to address some of the challenges mentioned above. Since Sheldon (1990) and Sheldon (1991), discussions have emerged that target ranking and rating (McKercher, 2005; Pechlaner, Zehrer, & Abfalter, 2002; Pechlaner, Zehrer, Matzler, & Abfalter, 2004), dissemination to practitioners (Frechtling, 2004), geographic distribution, author and institutional considerations (Jogaratnam, Chon, McCleary, Mena, & Yoo, 2005; Pearce, 1992; Ryan (2005b) and Ryan (2005a)). Our intention is not to repeatwhat has been accomplished in these works. Rather, the purpose here is to provide a critical analysis of where we are today, identify some recurring misunderstandings and confusions, and propose new innovations in ranking, rating, and disseminating tourism knowledge. Korobkin's quote above indicates, as we discovered, that lively debates over ranking methods and citation analysis have been going on in other disciplinary concentrations. Useful insights and directions may be gained by drawing on these to inform the interdisciplinarity of tourism studies.

A lively number of ranking and rating activities are being debated or pursued in various disciplines and fields in an attempt to effectively organize knowledge and methodologies within each one. The paper sketches out some of the imprecisions and inherent inconsistencies in various methods such as journal impact factors and citation analysis. It then goes on to show additional problems in evaluating journals and articles in tourism studies, including the subjectivity and deficiencies inherent in various databases. The macro-analysis points to the need for alternative schemes and innovation addressing both substantive as well as process issues. Drawing from other fields and our own (tourism studies), we argue that current article, author and journal impact factor calculations need to be revised to better reflect the diversity and legitimation of the growing knowledge base in tourism studies. Rather than a universal ranking system that generates a single list of journals, our analysis argues for the importance of delineating different types of journal orientations (such as theoretical, empirical and applied/practical journals) and specializations (disciplinary and topic-related). More specifically, we have argued that tourism journal ranking should reflect the scope of the field: the impacts of multiple disciplines shaping the journals and articles published in tourism journals, as well as the impact of tourism-related publications in other disciplines, sub-specialized areas and topics. This is especially important in the interdisciplinary area of tourism studies, which encompasses numerous sub-areas (e.g., hospitality as distinct from tourism studies), specializations such as ecotourism, heritage tourism, and events tourism. As noted in the previous section, it may be also be helpful to distinguish a journal's influence and quality from its theoretical and practical relevance to the field. We therefore join other authors in arguing that articles should be judged by their usefulness to a field, not by the prestige of the journal nor by the number of citations or hits ( Barman et al., 2001; Jones, 2002; Saha, Saint, & Christakis, 2003). Delineating the scope, influence, quality, and relevance of tourism journals using a scoring system agreed upon by a consensual scientific committee may be more helpful to identify the usefulness of an article than hits or citation frequency; moreover, the existing universal ranking systems used within databases vary greatly in the journals they rank ( Table 1). Academic departments and libraries could use these various lists to develop specific guidelines for their own particular situations (like hiring interdisciplinary faculty and addressing tenure and promotion decisions, library acquisitions, etc.). Table 2 could be expanded to include the spectrum of tourism journals, and differentiated under the categories of scope, influence, quality and relevance. Cleary, process mechanisms are needed to address the political issues that discussions of knowledge, quality and relevance necessarily involve. Crucial to the development of an effective ranking system in the interdisciplinary area of tourism studies is the consensus of the scientific community in this area on what constitutes good research and knowledge contribution, what theoretical and methodological practices guide researchers, and how these translate into practice. 6 Lacking at present, and badly in need of leadership action, is an annual international conference that convenes and enables such an interdisciplinary community in tourism studies. Most liberal arts and social science study areas like Management, Marketing, Urban Planning, Geography, Anthropology, Philosophy, Psychology and Sociology have well-established annual conferences associated with leading journals and associations in their field. Some offer international, national and regional conferences (a few even offer sub-streams related to tourism). In the young field of tourism studies, interdisciplinary fragmentations exacerbate the challenge of organizing communal spaces for its scientific community of scholars. Disciplinary and regionally focused conferences like the ones mentioned above, and other well-established ones like CHRIE, CAUTHE and TTRA (along with their related associations), are being supplemented by new annual conferences catering to special interests such as the tourism and literature conference organized in 2004 by the Centre for Tourism and Cultural Change (which also publishes the Journal of Tourism and Cultural Change), wine and food conferences, etc. Initiation and support of greater web-based activities and actions could also help improve the production, permeability and diffusion of knowledge. Creative and innovative use of the World Wide Web and the Internet is needed, with action directed to areas such as: • Increased use and acceptance of diverse online journals. Imperium, for instance, is a refereed online journal that tackles the postcolonial experience within the context of media and postcolonial studies. It does not seek copyright on papers posted to its site, but allows the author to continue shaping the paper for publication elsewhere (www.imperiumjournal.com. Accessed 9/10/05). • Use of web citations (in comparison to, or even replacement for, bibliographic citations (Vaughan & Shaw, 2003). • Online resources like the electronic tourism resource eRTR that attempts to link tourism researchers and practitioners (knowledge transfer—disseminating current research through this network). The short articles posted on eRTR are not peer-reviewed, expediting time from submission to posting. • Developing a semi-universal database (see Cameron, 1997) for tourism studies. CAB International is abstract only, Ingenta and google.scholar.com are useful but not interdisciplinary enough (yet), and not easily accessible in some locations. Centers with extensive research and reference databases (e.g., C.I.R.E.T., see www.ciret-tourism.com) could be valuable aides in undertaking this challenge (an impossible sounding task, admittedly). • The role of Google Scholar (http://scholar.google.com) should not be underestimated in undertaking the above task, and in offering alternative yet possibly more legitimate and complete databases of citation analysis. Appendix A provides a brief overview of this not-to-be-ignored force in the ranking and rating debates. • Expanding the role of the CrossRef initiative, which offers a unique reference linking system as noted in Appendix B. The idea behind CrossRef is very intriguing and could make the use of citations in rating/ranking journals and articles much more meaningful, especially as more publishers and their content is registered with the system. To summarize, we feel that the use of citation analysis and impact factors in tourism research will not become fully legitimate until much more comprehensive databases of citations are created and well-differentiated ranking/rating lists are constructed for numerous topics, specialties, etc. The ability to query such databases without restriction would also allow specific groups to create their own ranking/rating lists, which could be much more valuable and useful than the generic universal lists that are now generated. The CrossRef initiative and the advances being made by Google with Google Scholar offer some hope of achieving these goals. The field of tourism studies awaits such innovations. At the same time, academic leadership is needed to effectively organize the diverse knowledge contributions to this interdisciplinary area of study and practice. Academic leadership means more than the capability to communicate research in accredited journals. It means leadership in developing research programs and mentoring graduate students to achieve excellence in their chosen study area. It involves striving to push the boundaries of knowledge and learning in the field beyond basic applied research and development of technical tools, towards tackling interdisciplinary, critical and theoretical issues that contribute to knowledge (and consequently advances in applied technologies and tools). It also involves developing ranking and rating methods that are more sensitively tuned to the interdisciplinary challenges of tourism studies. This requires developing new interdisciplinary networks and discursive structures for dealing with knowledge-power (Foucault, 1980) in tourism studies.