افسانه مهارت حرفه ای خرید. شواهد بیشتر در مورد اینکه آیا کامپیوترها می توانند در خصوص تأمین تجهیزات تصمیمات بهتری بگیرند

In a previous experiment, we have shown that risk assessments of purchasing experts are certainly not better than that of subjects untrained in purchasing, and worse than the decisions made by formal models (J. Purchas. Supply Manage. 9 (2003) 191–198). Since both these results are rather counterintuitive, we conducted a series of experiments geared at replication and extension of these findings. These new experiments show that our previous results are robust, and reveal an additional finding that is both worrying and puzzling. It actually seems to be the case that for the purchasing decision tasks in our experiments, experts perform worse with growing experience. It therefore seems that, at least for the kinds of purchasing decisions under study, it does not make much sense to use expert judgments at all. However, we show that there is a way in which expert judgments can be used in combination with formal models to improve the predictive accuracy of purchasing predictions. In our case, superior predictions are made when we combine the prediction of a formal model with the prediction of the ‘average expert’, thereby combining the robust linear trends as encapsulated in the formal model with the more intuitive configural rules used by experts. We provide several explanations for this phenomenon.

Most people would agree that at least one of the tasks of a purchase manager is to decide which of a set of purchasing transactions needs purchase management more. For instance, for some transactions it makes sense to ask for many tenders, invest much in the screening of suppliers, involve much time in negotiating, and put a serious effort in writing a detailed contract. For other transactions such investments are not effective or not efficient (Batenburg et al., 2000). Although it is typically part of a purchase manager's job, there are compelling arguments on the basis of the literature on clinical versus statistical prediction that suggest that purchase managers—like all other humans—are typically not good at making precisely these kinds of judgments. In a review study, Grove et al. (2000) have shown that for single, quantitative decision tasks computer models almost always perform at least as good as or better than human experts. Most of the studies they reviewed were based on tests in medical, forensic and clinical-personality studies (102 out of 136). There are only a handful of studies comparing human experts with models that deals with ‘more economic topics’. Grove et al. (2000, Table 1) mention studies on business failure, job performance, job turnover, business startup success, job success and work productivity. In a previous publication ( Snijders et al., 2003), we set out to test this assertion when it comes to judgment and decision-making in purchasing, and reported on an alarming and somewhat counterintuitive result. It indeed turned out that, for the cases under study, purchasing professionals do not make better purchase decisions than undergraduates, and both are actually outperformed by a computer model (using a simple formula). Table 1. Spearman correlations between actual and predicted scores, averaged per group, as in Snijders et al. (2003) Formula 0.37 Students 0.26 Purchasing professionals 0.24 Table options Though in principle this result is in line with research in other areas, and in that sense perhaps need not be treated as a surprising finding, we want to analyze some of the evident follow-up questions related to this result. First, we consider how robust this finding is. Perhaps our finding was simply for a purchasing professional rather unlucky statistical fluke, so we tried to replicate our findings (and succeeded). Second, we extended our previous research and considered possible reasons for our findings by looking at the way in which experts, in general, tend to behave. Finally, we focus on whether and how the decision-making of purchasing professionals can be improved, based on the literature on experts and expertise.

The finding that a formula predicts the prevalence of problems better than purchasing professionals has been shown to be a robust finding across different experiments. Our analyses and review of part of the relevant literature also give some feel for the causes underlying the relatively poor performance of professionals. Experts process a judgment task differently than do non-professionals, tend to relate their judgments to selectively available cases, and use larger chunks of information at the same time. Even though they do often not outperform less experienced colleagues, they are generally more certain about the accuracy of their outcomes. When looking closer at the data, we even find some evidence that over the course of their career and with increasing experience, purchasing professionals develop habits that hamper rather than facilitate providing accurate predictions. An accurate overall prediction typically involves consistently weighing the available case characteristics, and professionals are less inclined to do that as their experience progresses. One might be tempted to think that at this point one cannot but conclude that the judgments of purchasing professionals are superfluous altogether. And undeniably, most potential improvements of the judgments of professionals do not lead to judgments that outperform the cross-validated formula. However, we do find evidence that a sensible combination of both the formula and the expert judgments may improve upon the professional and may even outperform the formula. This involves first combining the expert judgments into an ‘average expert’, and then combining this with the formula. Taken together, our analyses show that purchasing professionals are not an exception to the general rule in the scientific literature that computer models outperform professionals when it comes to the kind of clear cut decision tasks as described here. Though this may seem to paint a bleak picture of the abilities of purchasing professionals, one should not stretch our findings beyond their limits. Let us first stress that we are not claiming that—in general—computer models are better purchasing professionals. There is much more to being a purchasing professional than just making single shot decisions of the kind we discussed, and certainly many of the challenges purchasing professionals have to deal with are beyond the realm of a computer model. Our conclusions are confined to tasks of the kind as in our experiment. These are tasks where the decision is clear-cut (e.g., Should I use tenders and how many?), and where there is at least some data on past performance available for a model to be constructed. We discuss some of these issues in the section on practical implications below. Several factors typically make such decision tasks more difficult for purchasing professionals as opposed to computer models: they work in an environment where immediate and precise feedback is generally lacking, and where many factors influence the eventual outcome so that the optimality of their own behavior cannot be clearly distinguished. Several counterarguments to our results can be thought of. First of all, in our experiment the professionals are forced to make their decisions in a context that is certainly more abstract than they are used to. Although our results still hold if we only use the data of the professionals who claimed to be certain about their answers, it is possible that their judgments would improve and perhaps surpass the judgments of the model with increasing information about the transaction. We aim to test this in a future experiment. A second counterargument we can think of is that in our experiment there are no real incentives to make the right decisions. Purchasing professionals are used to decide in situations with a lot of money at stake, and are perhaps less inclined to think carefully when all there is to gain is a high test score. It is difficult to argue against this given available data. Closer inspection of the cases where professionals took the test more than once shows that their results do not tend to improve on second runs. This could perhaps be interpreted as a sign that extra attention does not help much, but we admit that this is not very compelling. Again, this is something to take into account in a future experiment.