بررسی سیستماتیک مقالات تحقیقات نرم افزاری هزینه کیفیت

Software quality costs have not received as much attention from the research community as other economic aspects of software development. Over the last three decades, a number of articles on this topic have appeared in a range of journals, but comprehensive overviews of this body of research are not available. For the detailed review of software quality cost research presented in this article, we collect 87 articles published between 1980 and 2009 in 60 leading computing journals. We study the distribution of these articles across research disciplines and journals as well as over time. Moreover, we identify the predominant researchers in the software quality cost domain and the related research clusters. We also classify the articles according to three properties, namely, research topic, research scope, and research approach. This categorization enables us to identify aspects emphasized by previous research on software quality costs and to point out promising future research directions. Our review shows that prevention costs have gained the least attention, in spite of their big cost impact. It also reveals that only one article has targeted multiple companies. Further, we observe that many articles do not empirically validate their findings. This is especially true for those articles dealing with an entire firm.

For decades, users of software solutions have been suffering from poor solution quality (Whittaker and Voas, 2002). Over the years, quality has emerged to be a key issue in software development (Prahalad and Krishnan, 1999). Software vendors have attempted to tackle this issue by adapting concepts from other engineering disciplines, such as manufacturing (Antony and Fergusson, 2004). There, approaches ranging from Total Quality Management over Six Sigma and Kaizen to Lean Production have led to significant gains in productivity and quality. To attain similar results in software development, many of these concepts have been adapted and tailored to its characteristics (Middleton and Sutton, 2005). In this quest for higher productivity and quality, the economics of software engineering are of particular interest (Boehm, 1981). While some economic aspects, such as software development effort estimation (Jorgensen and Shepperd, 2007) and software process improvement (Hansen et al., 2004), have frequently been discussed, others have received less attention. Indeed, little research has specifically been devoted to those costs which are “incurred in the pursuit of [software] quality or in performing quality-related activities” (Pressman, 2010, p. 407). This is remarkable, because software vendors typically spend 30–50% of their development budget on defect detection and correction (Ebert and Dumke, 2010). In most engineering disciplines, literature studies summarizing the latest research results on quality costs are regularly published (e.g., Williams et al., 1999 and Schiffauerova and Thomson, 2006). To the best of our knowledge, this is not the case in software engineering; no review prior to ours has in particular been devoted to software quality cost research. However, several studies published in the broader field of software quality and software economics have also covered some quality cost aspects. For instance, the survey of software quality assurance research by Rai et al. (1998) considers software quality costs among other economic aspects, and Jorgensen and Shepperd (2007) systematically review work on software development effort estimation including approaches applicable to software quality cost estimation. This article tries to close this research gap. It is exclusively devoted to software quality cost research. Our objective is to systematically review and structure the existing body of research on software quality costs and to identify areas for future research. We analyze 87 articles published between 1980 and 2009 in 60 leading computing journals for answering eight research questions. These research questions are directed at the research domain in general as well as at specifics of the existing research, regarding the software quality cost categories examined, the scope of investigation, and the research approaches employed. The main contributions of this systematic literature review are thus two-fold: (1) We systematically gather and discuss domain-relevant articles, covering 30 years and a large number of computing journals. (2) By answering our eight research questions, we identify aspects emphasized by prior research and areas that future work should address. The remainder of this article is structured as follows: In Section 2, the eight research questions of our systematic literature review are presented. Next, Section 3 introduces the review method applied and the classification scheme used. The results of the review are then discussed in Section 4, answering the research questions formulated. The article closes with Section 5, which sums up our findings and suggests areas for future research.

The main goal of this systematic literature review on software quality costs was to structure existing work in this field and to guide researchers to promising future research directions. Our results have revealed that software quality cost research is mostly published in software-engineering-related journals. While only the IEEE Transactions on Software Engineering have published more than ten articles, articles on software quality cost research have appeared in as many as 31 journals. We have seen that 12 authors have been involved in at least two of the articles examined. These authors account for a considerable fraction of all publications covered in this review. They and the seven related research clusters shape the research domain by studying software quality costs from different perspectives. Regarding the content of the software quality cost studies, we proposed three properties (namely, research topic, research scope, and research approach) and categorized all identified articles. We found that appraisal and failure costs are often analyzed jointly. This may be due to the direct link between these kinds of costs: Failure costs tend to increase when less effort is spent on appraisal activities, and vice versa. There is no such direct interdependency with prevention costs, which are related to long-term investments like process improvement initiatives. Therefore, prevention costs can easily be analyzed separately from the other cost types. We also found that software quality cost research has primarily been carried out by means of model building and theory generation. While the community has thus developed a sound understanding of the research domain’s structure, empirical validation is often lacking. Only about a third of the analyzed articles presents a case study or more extensive empirical results. This appears to be insufficient for software quality cost research, which strongly relies on quantitative data to generate new findings. There is thus a need for novel approaches to gather quality cost data, as well as stronger cooperation between industry and research to make such data available. Further, our classification of all articles has unveiled some interesting dependencies between the three properties. For example, we observed a link between the research scope and the software quality cost categories studied. While at the company level, prevention costs are sometimes investigated, the more specific project/product and activity levels concentrate on appraisal and failure costs. This finding can be explained by the fact that prevention activities typically occur in form of process improvements, which cannot be assigned to a single project. Overall, prevention costs have received the least attention, although the highest quality cost savings can be achieved by avoiding defects in the first place via investments into preventive activities. Consequently, it seems to be suggestive to put more focus on this cost type. The same holds for research on the industry level. We are only aware of one article that gathers industry-wide data and benchmarks software companies by their quality costs. Regarding software quality cost modeling, our review provides a mixed picture: While many models have been proposed at the project/product level and the activity level, there is no article suggesting a comprehensive model at the company level (or at the industry level). Only such comprehensive models including prevention, appraisal and failure costs might give a holistic view on software quality costs as well as insights into the right balance between these three cost types.