رویکردANP برای ارزیابی پروژه R & D بر اساس وابستگی های متقابل بین اهداف تحقیق و معیارهای ارزیابی
|کد مقاله||سال انتشار||مقاله انگلیسی||ترجمه فارسی||تعداد کلمات|
|6141||2010||8 صفحه PDF||سفارش دهید||6552 کلمه|
Publisher : Elsevier - Science Direct (الزویر - ساینس دایرکت)
Journal : Decision Support Systems, Volume 49, Issue 3, June 2010, Pages 335–342
As countries strive to become more efficient in investing limited resources in various national R&D projects, the evaluation of project has become increasingly important. However, due to the heterogeneity of the objectives of national R&D programs, few studies have been conducted on comparing programs on the basis of performance. This study explores the application of the analytic network process (ANP) approach for the evaluation of R&D projects that are elements of programs with heterogeneous objectives. The ANP produced the final priorities of projects with respect to benefits and costs when there are interdependencies between programs and evaluation criteria. The paper provides value to practitioners by providing a generic model for project evaluation. For researchers, the paper demonstrates a novel application of ANP under specific situation of heterogeneous objectives. The ANP approach is tested against empirical data drawn from R&D projects sponsored by the Korean government.
As R&D is a major force driving national competitive advantage, governments in many countries have increased the level of R&D investment by sponsoring R&D projects. Due to the scale of funding, the complexity of technology, and the heterogeneity of objectives, the evaluation of government-sponsored projects is usually viewed as a multiple criteria decision-making (MCDM) problem. A large number of methods and techniques exist in the literature for the application of multiple criteria approaches to R&D project evaluation . Approaches tend to be either qualitative or quantitative, ranging from unstructured peer review, which is normally made by a review committee with experts from academia, industry, and government, to sophisticated mathematical programming, including integer programming, linear programming, nonlinear programming, goal programming, dynamic programming, and portfolio optimization. Overviews on the topic of R&D project evaluation may be found in Henriksen and Traynor , Martino , and Steele . Most studies in R&D project evaluation are in the private sector. However, a government-sponsored project differs from those in the private sector because government-sponsored R&D project is by nature a strategic and long-term investment. Government investment in R&D seeks to influence the private sector to make investments in technological fields important to the country as a whole. Investments led by government therefore address issues of national policy that are promulgated through a variety of outcomes, such as publishing academic papers for basic research field, issuing patents and developing prototypes for applied research field, and providing funds and researchers for R&D human resource development field. This variety of outcomes is due to the heterogeneity of the objectives of national R&D programs. Table 1 represents various national R&D programs in Korea and their objectives. Each program is composed of several R&D projects. This heterogeneity of the objectives of national R&D programs makes it intractably difficult to compare the relative performance of various government-sponsored R&D projects which belong to different programs. Consequently there have been few attempts to measure, at the same time and in the same context, the performance of several R&D projects that are elements of different government-sponsored programs.The performance of a R&D project should be measured based on the unique characteristics of R&D programs to which the projects belong. Also, to incorporate several input/output variables of a project and produce a single measure for performance comparison, the relative importance of variables needs to be determined and fixed. This study presents an evaluation decision model with multiple criteria that assesses R&D projects that are elements of R&D programs with heterogeneous objectives. In this paper, it is assumed that interdependency exist between programs (Table 1) and evaluation criteria (Table 2). For example, the importance of a certain program may vary depending on which criterion (for example, academic journal publication) is considered because each program is initiated with its own main objective. Also, the importance of a certain criterion may vary depending on which program is considered as well. Thus, this interrelationship between programs and evaluation criteria needs to be mirrored in evaluation of alternative projects which have their own program membership.The purpose of this study is to solve the interdependency issue between programs and evaluation criteria by using analytic network process (ANP). The ANP is a generalization of the analytic hierarchy process (AHP), which is one of the most widely used MCDM methods . AHP in general have however certain limitations since not every problem can be defined as a hierarchical model. It is true that an extension of AHP allows the representation of more complicated relationships through the ANP . That enhances the expressive power of this MCDM methodology. The ANP allows for more complex interrelationships among factors because it replaces the hierarchy in the AHP with a network. It is therefore capable of providing priorities of projects that capture network relationships among factors such as programs and evaluation criteria. Most ANP applications in the literature deal with the interdependency among evaluation criteria themselves, and the alternatives are supposed to be in homogeneous set (in the same R&D program). However, it becomes a difficult task to determine the fixed importance of variables if the heterogeneity of different programs' objectives is involved. The aim of this paper is that, in order to overcome these limitations, we tried to incorporate the heterogeneous sets of alternatives and decision criteria by a wise use of ANP considering the interactions between the several decision criteria and several objectives of heterogeneous sets(programs) of alternatives(projects). This study also analyzes the costs and benefits associated with network relationships between fourteen kinds of R&D projects in various government-led programs. The resulting preference index supports efficiency in the development of national R&D policy. The remainder of this paper is organized as follows. Section 2 reviews the analytic network process (ANP), the methodology underlying the proposed approach. Section 3 describes the heterogeneous nature of Korean government sponsored R&D. Section 4 details the proposed approach and Section 5 reports the results of the application of ANP. Some concluding remarks are made in Section 6.
نتیجه گیری انگلیسی
This study proposed the use of the ANP approach for the evaluation of R&D projects that are elements of programs with different purposes. The case of Korean government sponsored R&D projects was presented to illustrate the proposed evaluation approach. Results of the case analysis demonstrate that the ANP produced the final priorities of projects based on benefit and cost that take into account the interdependency between programs and evaluation criteria. The paper provides value to practitioners by providing a generic model for project evaluation, and to researchers by demonstrating a new and novel application of ANP. While we believe that the model presented is meant to be a generic model applicable across different R&D project evaluation environment, it is acknowledged that the levels of decision criteria and the decision makers' decision would be different depending on the environment involved. Actually, this is one of the strengths of ANP: the applicability of general methodology to a specific situation. Depending on the decision environment, additional factors and criteria could be added. Thus, the modification of the proposed approach for more complicated modeling will be a promising area for future research. For example, the proposed ANP model only reflects the interdependency between program and criteria components, but there may be interrelationships between sub-criteria belonging to different criteria. Incorporating these relationships in the ANP model is expected to produce more realistic results. As far as the validation issue is concerned, proof that the ANP matches what thoughtful people do is validated through the many examples in the numerous literatures regarding ANP. There is no way to prevent someone from putting in poor judgments, which do not in fact faithfully reflect their feelings or the experience they are attempting to represent, and when this occurs the ANP does not give back good results. Thus it helps to have a group critique what an individual does to remind that individual about facts that may have been forgotten or point out judgments that may be wrong. The ANP makes it possible for people to debate and combine their judgments in order to draw a reasonable group decision. It opens the door for taking anyone's suggestion and including it in the considerations and giving it a high or a low priority in the end. This is also the beauty of the ANP model. Also, fuzzy set theory may prove effective in reducing the vagueness associated with manager preferences elicited via pair-wise comparisons. Efforts to exploit these avenues of future research will increase the value of the model presented in this paper as a decision-making tool. Appendix A. Brief description of ANP procedure and its sample application The following provides a simple application and description of procedures of the ANP model. As an example, ANP is applied to the problem of selecting an alternative for certain interest. There are three alternatives which are believed to be the most competitive: A1, A2 and A3. In this study, a simple feedback model is structured with three decision criteria such as C1, C2, and C3 as shown in Fig. A.1.The flow of influence in a feedback network model is specified by links. A link from one element, such as a criterion, to other elements, such as alternatives, specifies that influence can flow from the former to the latter. As seen in Fig. A.1, not only does the importance of the criteria determine the importance of the alternatives as in a hierarchy, but also the importance of the alternatives themselves determines the importance of the criteria. Each criterion in this example has a link to the three alternatives to indicate the flow of influence from the criterion to the alternatives. Pair-wise comparisons are made to determine the relative influence which the criterion has on the relative preferences of the alternatives. Similarly, each alternative in this example has a link to the three criteria to indicate the flow of influence from the alternative to the criteria. Pair-wise comparisons are made to determine the relative influence which the alternative has on the relative importance of the criteria. Once the feedback network model is set up, pair-wise comparisons will be made to indicate the relative amount of influence that flows from one element to each of the other elements. The way of conducting pair-wise comparison and obtaining priority vectors is the same as in the AHP. The relative importance values are determined with a scale of 1–9, where a score of 1 indicates equal importance between the two elements and 9 represents the extreme importance of one element compared to the other one. A reciprocal value is assigned to the inverse comparison; that is, aji = 1/aij, where aij denotes the importance of the ith element compared to the jth element. Also, aii = 1 are preserved in the pair-wise comparison matrix. Then, the eigenvector method is usually employed to obtain local priority vectors for each pair-wise comparison matrix. Saaty  proposes several algorithms for approximating the local priority vectors including an eigenvector method to derive the priorities, where each component in the principal right eigenvector is normalized between 0 and 1 to represent the relative priority of alternatives. Choo and Wedley  have discussed 18 approaches that attempted to achieve this aim. However, Golden et al.  have shown that the eigenvector approach is “a theoretically and practically proven” method for estimating the priorities. Thus, the eigenvector approach formally deals with human errors inherent in any decision-making processes. The estimate of the overall relative priority for an alternative pi is a component in a vector P. P is derived by solving the eigenvalue problem AP = λmaxP, where λmax is the largest eigenvalue of matrix A, A is the comparison matrix, and P is the principal (dominant) right eigenvector corresponding to the eigenvalue λmax. In this paper, the following three-step procedure is used to synthesize priorities: (a) sum the values in each column of the pair-wise comparison matrix. (b) Divide each element in a column by the sum of its respective column. The resultant matrix is referred to as the normalized pair-wise comparison matrix. (c) Sum the elements in each row of the normalized pair-wise comparison matrix, and divide the sum by n elements in a row. These final numbers provide an estimate of the relative priorities for the elements being compared with respect to other element. Three alternatives are compared with respect to the criteria C1, C2, and C3 in Table A.1. For example, with respect to C1, a reasonable judgment is that A1 is two times more preferred than A2 (in other words, A2 is half times more preferred than A1). However, with respect to C2, A1 is five times more preferred than A2.In Table A.2, three criteria are compared with respect to the alternatives A1, A2, and A3 in a similar manner with Table A.1. In Table A.2, for example with respect to A1, a reasonable judgment is that C1 is five times more important than C2 (in other words, C2 is five times less important than C1).Then the following supermatrix, whose concept is similar to a Markov chain process is constructed in Table A.3 by using the vector weights for the alternatives with respect to each criterion and the vector weights of the criteria with respect to each alternative and it is used to assess the results of feedback network model.The final priorities for both the criteria and alternatives are obtained by multiplying this supermatrix by itself numerous times until the columns stabilize and become identical in each block. For this convergence to occur, the supermatrix needs to be ‘column stochastic’. In other words, the sum of each column of the supermatrix needs to be one. Raising the supermatrix to the power 2k + 1, where k is an arbitrarily large number, allows convergence  and . As the supermatrix in this example is already column stochastic, it is directly transformed into the limit supermatrix. The limiting power of the supermatrix in this example is reached at the fifth stage as in Table A.4:In the last three rows of this limit supermatrix the limiting priorities of the three alternatives are seen. The priorities are in the region of 0.563 0.260 and 0.177 for A1, A2, and A3 respectively. As a result of this illustrative example, A1 can be said to be a more appropriate alternative than A2 and A3.