بهبود روند دلفی : درسهایی از تحقیقات اجتماعی روانشناختی

The Delphi technique was largely developed to avoid the problems of freely interacting groups such as dominant individuals and pressure to conform to the majority view. Our review of the social psychological literature reveals some obstacles to Delphi achieving its full potential relative to other cheaper and easier methods of aggregating judgment. We identify residual normative and informational pressures towards consensus that potentially reduce process gain that might otherwise be achieved. For instance, panelist confidence may act as a signal of status rather than be a valid cue to expertise, whereas consensus appears to have a strong influence on the final outcome that can reduce its accuracy when there are valid minority opinions. We argue that process gain in Delphi must occur through those further from the "truth" changing their opinion more than those closer to the truth, with the general direction of opinion change being towards the truth. For such virtuous opinion change to occur we suggest the need to both facilitate opinion change and provide good cues as to where the truth lies. Research on Judge Advisor Systems shows that people usually do not change their opinion as much as they should, giving too much weight to their own opinion and too little to the views of others—this bias can be reduced by increasing involvement and motivation. In addition, we propose that the best way to provide good cues as to the direction of the truth is to elicit rich reasoning from panelists about the judgment or choice in question, then use this as feedback. We suggest practical ways of focusing and deepening panelists’ consideration and evaluation of such reasoning—such that all proffered opinions are well-evaluated. Additionally, we propose a model of opinion change in Delphi for use as a paradigm for future process-orientated research.

The Delphi technique was largely developed in order to avoid the problems of freely interacting groups—such as dominant individuals and pressure to conform to the majority opinion—these sources of “process loss” [1] are removed through anonymization and controlled feedback [2], [3] and [4]. Further, repeated rounds of polling and feedback permit “process gain” as experts converge on the truth,1 while averaging removes error [5]. But is the process gain the best that could be achieved? Studies suggest that Delphi usually does better than freely interacting groups, or simply taking the average of initial opinion (see e.g. [5] for a review). However, the advantage of Delphi is more equivocal when compared to other “nominal group techniques” that anonymize, but do not use repeated rounds of polling and controlled feedback, and thus are much cheaper and easier to carry out. The questions we wish to address here are whether the Delphi technique is achieving its full potential as a means for improving expert judgment and decision making and, if not, whether there is any scope to modify the Delphi technique so as to enhance its outcomes?2 Rowe, Wright and McColl [6] argue that the key to answering such questions as these is to look at the processes by which Delphi facilitates opinion change in groups—and in particular, to investigate opinion change that might lead to improvement in accuracy. We concur with Rowe et al. and in this paper we will look in detail at such processes through the lens of relevant literature on the psychology of groups—in other words, social psychology. Once we understand these processes we can then see whether there are any barriers in traditional Delphi to achieving the maximum accuracy improvement possible—relative to any less costly methods (in terms of time, money, effort etc)—if such barriers are found we will devise means for removing them. To these ends we will develop a model of opinion change in Delphi.

As is clear from the papers in this Special Issue, more than 50 years since its introduction to the public, Delphi is still a widely-used technique that brings many benefits to users both in the public and private sectors. However, our review of the literature, mostly originating from social psychology, has revealed that there remain some obstacles to Delphi achieving its full potential in terms of outcomes—relative to other cheaper and easier methods of aggregating judgment. We identified residual normative and/or informational pressures towards consensus that may operate to potentially reduce process gain that might otherwise be achieved from the iterative feedback that sets Delphi apart from other nominal group techniques. In particular, panelist confidence, both in absolute terms and relative to that expressed by other panel members, may be used as a cue to expertise although evidence suggests it might be a poor indicator of true expertise and may actually be used as a normative signal of status. Further, consensus or plurality of opinions appears to have a strong influence on the final judgment or decision that can be particularly detrimental to the accuracy of the outcome when there is valid minority or outlying opinions. In the light of this discussion we proposed that confidence should not be fed back to panelists and wherever possible, consensus and plurality should be disguised in the feedback. On the basis of our literature review we also argue that Delphi process gain must occur through virtuous opinion change—that is to say, that those further from the "truth" change their opinion more than those closer to the truth, and the direction of opinion change tends to be towards the truth. For such virtuous opinion change to occur we argued that it is necessary to (a) facilitate opinion change and (b) provide good cues as to where the truth lies. It has been shown in research on Judge Advisor Systems that often people do not change their opinion as much as they ideally should, giving too much weight to their own opinion and too little to the views of others—a phenomenon known as egocentric discounting. This work suggests that egocentric discounting can be reduced, and therefore opinion change encouraged, by increasing involvement and motivation—we therefore recommend offering incentives, financial or otherwise, as appropriate and increasing panelists’ accountability and feelings of participation; these actions may have secondary benefits in terms of reduced drop-out rate by panelists and willingness to take part in future Delphi exercises. In addition, we proposed that the best way to provide good cues as to the direction of the truth is to (i) elicit rich reasoning from panelists about the judgment or choice in question, and then (ii) to feedback this reasoning to participants in a form designed to minimize the forces of social influence described above. Finally, we proposed ways of avoiding letting this rich feedback simply further reinforce majority opinion at the expense of minority views by creating heterogeneity of opinion through initial selection of panel members then by use of role playing, devil's advocacy, and dialectical enquiry. In these ways, the reasoning underpinning proffered opinions can be highlighted and analyzed by Delphi panel members. Of course, our proposals need to be backed with additional research both of a pure and an applied nature. Very little research has been conducted directly on the processes of opinion change in Delphi, which is why we turned to work in social psychology. In this paper we have proposed a model of opinion change in Delphi that could be a starting point for such future research. In particular, the relative contributions of the various influences on opinion change specified in the model need testing for different kinds of task and panel composition, as do interactions between these variables. In addition, our practical recommendations for enhancements to the Delphi technique also require testing in order to see which if our advocated modifications do provide significant improvements in the Delphi yield—compared to more standard Delphi practice.