Software process improvement (SPI) is a widely recognized approach that software companies implement to improve quality, productivity, and time-to-market. Assessing and analyzing performance improvements are important SPI activities. However, many SPI managers have found it difficult to develop and implement effective performance measurement programs for SPI, in part because guidelines for conducting SPI measurements are scarce. We address this gap in the SPI literature by examining major problems that SPI change agents encounter when developing and implementing SPI measurement programs. We report on a longitudinal study of an SPI change initiative and the challenges that the SPI Team faced in dealing with the issues of measuring effectiveness of the initiative. We systematically analyze an SPI performance measurement program to understand its limitations and the problems that the SPI Team encountered when implementing it. We used an organizational change theory framework to derive theoretical and practical insights that can help managers and researchers develop and implement better SPI performance measurement programs.
The last decade has been one of continuous and unrelenting global competition in the software industry. A growing lower wage and a highly skilled software engineering labor force in India and other Asian countries have precipitated shifts in software production from Europe and North America to Asia. In response to these challenges, senior managers in European and North American software industries are adopting a variety of strategies to stay competitive. For example, some firms are outsourcing software development to Asia, and others are setting up Asian offices to take advantage of the lower cost of production. Still others are implementing various forms of organizational change, such as business process re-engineering; more flexible approaches to organizing work, such as teams and autonomous work-groups; and software process improvement (SPI).
Many business firms view SPI as a strategic issue and are involved in organizational change initiatives to improve their software development practices. The fundamental objective of the SPI movement is to change the organizational practices of software development firms, in order to improve software quality and reliability, employee and customer satisfaction, and profitability (Hansen, Rose, & Tjørnehøj, 2004). One of the factors that affect the success of an SPI program is the presence of an effective measurement program to support decision-making and inform managers about the state of their software development practices. The measurement and analysis of organizational effectiveness is an important activity in any SPI initiative. Measuring aspects of organizational effectiveness in software development can provide management with critical information upon which to base decisions about allocating resources to effect change in specific areas.
Although there is broad agreement that a systematic and reliable methodology for measuring effectiveness is necessary for the success of SPI change initiatives (Goldenson, Emam, Herbsleb, & Deephouse, 1996; Goldenson & Herbsleb, 1995; Hayes & Zubrow, 1995), there is little agreement on what should be measured. Many researchers view the lack of a systematic and reliable measurement methodology as a major factor contributing to the high failure rate of SPI initiatives. For example, Bill Curtis, one of the authors of the Capability Maturity Model (CMM) (Paulk, Curtis, Chrissis, & Weber, 1993a), estimated that as many as 70% of the firms implementing SPI are unsuccessful (Curtis, 1996). More research is needed to develop systematic measurement programs for assessing effectiveness of SPI initiatives. The main objective in this paper is to develop an understanding of how to measure software development performance and how these measures can inform software development managers, SPI practitioners, and senior management.
The theoretical foundation of our analysis is organizational change theory, and our research methodology is based on longitudinal process analysis. The focus of our discussion is a longitudinal study of an SPI change initiative in the Danish software company, Danske Data. The rest of the paper is organized as follows: The next section describes the research methodology. Section 3 describes the organizational change theory framework we used in the study. Section 4 describes the case organization and the SPI program that is the subject of this study. Section 5 presents the empirical findings, and Section 6 outlines implications for future research on measuring effectiveness in SPI.
