Both the design and use dimensions of Performance Measurement Systems (PMSs) have been widely explored by managerial and accounting scholars (Broadbent and Laughlin, 2009, Ferreira and Otley, 2009a, Li and Tang, 2009 and Otley, 1999). A PMS is defined as a set of mechanisms and processes used by an organisation to identify key objectives and support the implementation of actions, planning, measurement, control, rewarding and learning (Ferreira & Otley, 2009a). Research on PMS design has mainly sought to investigate how to define the PMS best suited to an organisation’s characteristics, with a view to augmenting performance effectiveness (see Chenhall, 2003 for a review). To this end, such studies have analysed the impact of organisational and contextual variables on the design of the PMS, and its effects on performance outcomes. More recently, the attention of scholars has been catalysed by the topic of PMS use, with researchers striving to understand how managers use, or might use, the information provided by a PMS (Nilsson and Kald, 2002, Tuomela, 2005 and Wouters, 2009), and the determinants of these different uses (Henri, 2006a, Hoques and James, 2000, van Veen-Dirks, 2010 and Verbeeten and Boons, 2009).
The above studies have investigated the determinants of PMS design and use separately, finding evidence that both these dimensions, here conceptualised as “what” and “how”, influence the operation of the PMS. Yet, though it is recognised that the diversity of measurement and their use are “two closely intertwined dimensions of PMSs that must be examined specifically” (Henri, 2006a, p. 97), research to date has overlooked the interaction between them. PMS design has recently been investigated jointly with its use (see for example de Haas and Kleingeld, 1999, Ahrens and Chapman, 2004, Henri, 2006a and Ferreira and Otley, 2009b). These contributions indirectly tackled the issue of interaction between design and use, but with mixed findings. For example, de Haas and Kleingeld (1999) propose a normative framework for designing a PMS that enables a specific type of use, namely interactive. Using a longitudinal case study in a restaurant chain, Ahrens and Chapman (2004) show how specific design principles foster an enabling PMS use, allowing managers to simultaneously pursue efficiency and flexibility. These studies suggest that the “design of organisational controls play a crucial role in fitting organisational behaviour to organisational goals” (de Haas & Kleingeld, 1999, p. 234), thereby treating design as the predominant factor shaping the appropriateness of a PMS. In contrast, other case studies acknowledge that “the use of the Management Control System (MCS) has the ability to counteract issues with MCS design” (Ferreira & Otley, 2009b, p. 35). For instance, Henri (2006a) found that the way in which a PMS is used entrains differences in performance measures. More specifically, a monitoring use is associated with a predominance of financial measures, while a strategic use is mainly characterised by a predominance of non-financial measures. Ferreira and Otley (2009b) instead emphasise the prominent role of use, highlighting how intensity of PMS use can counterbalance misfits in PMS design. These mixed findings might be explained by the fact that the studies in question simultaneously investigated PMS design and use, but overlooked their interaction.
Building upon “loose” contingency (Chapman, 1997), the present work has the objective of theoretically and empirically assessing how the aspects of PMS design and use are interrelated, through a focus on a well-known performance measurement technique, the BSC (Kaplan & Norton, 1992). Use is specifically investigated by adopting Simons’ (1995) distinction between diagnostic and interactive control, while the design dimension is analysed with reference to four attributes emerging from the previous BSC literature (e.g. Malmi, 2001 and Speckbacher et al., 2003). The balance between financial and non-financial performance measures, the cascading of the BSC, the determination of target levels and the reward system associated with BSC measures are here related to how the PMS is used, whether diagnostically or interactively. The choice of a “loose” contingency approach was driven by the need for more accurate results than those obtainable from “hard” contingency testing (Chapman, 1997), thanks to a closer contact between the organisation and the researcher (Otley, 1980). Accordingly, the research question investigated in this study is the following: how is the type of use, characterised as diagnostic or interactive, related to the BSC design? We empirically address this question in seven non-financial firms that had been using a BSC for at least three years. Through an in-depth investigation, we identify two distinct types of Balanced Scorecards, which we call Diag-BSC and Int-BSC, associated with different styles of use (diagnostic or interactive), and different design features.
The results of our research are discussed in the remainder of this paper. First, we present previous studies on the design and use of PMSs; we then go onto operationalise the concepts of design and use specifically for the BSC, with the purpose of explaining how we detected these dimensions in the field. Next, the research methodology is explained, followed by a discussion of the results of the case studies. Finally we draw some conclusions.
The antecedents and the effects of PMS design and use have been receiving continued attention in managerial research, but recent contributions suggest a need to explore how these two concepts are interrelated (e.g. Henri, 2006b). Drawing on a multiple case study, this work explores the interaction between BSC design and use, conceptualising PMS design as the “what” dimension and PMS use as the “how” dimension. The “what” embraces the technical features managers choose when implementing a PMS, while the “how” concerns the control style associated with the information provided by the PMS. Our findings indicate that such an interaction indeed exists: BSCs used diagnostically are characterised by a prevalence of financial measures, absence of cascading, explicit targets, and no link with the reward system. Int-BSCs instead have the opposite traits: an evenly balanced set of financial and non-financial measures, cascading of the BSC, implicit targets, and a link to the reward system.
These results offer some useful insights for both academics and practitioners. From an academic perspective, we answer the call for investigating design jointly with use (Henri, 2006a and Ferreira and Otley, 2009b), highlighting the interaction between the “what” and the “how”, and responding to the more general call for bringing a holistic perspective to the study of PMS (Ferreira & Otley, 2009a). Focusing specifically on the BSC, our study shows that its design features differ depending on the specific style of control, whether diagnostic or interactive. This finding supports the claim that design and use are closely intertwined (Henri, 2006a), but more specifically it provides insights on how the characteristics of design are associated with an interactive or diagnostic control style, suggesting a set of hypotheses for “hard” contingency and testing. The first hypothesis predicts that BSCs characterised by a predominance of non-financial KPIs will be associated with a more interactive style of control, as advanced by previous work (e.g. Fried, 2010). The second hypothesis suggests that there is a relationship between implicit targets and an interactive control style, as advocated by Simons (1995) and by other scholars investigating the participative approach to budget setting (Libby, 1999). The third hypothesis is that an interactive style of use will feature links with the reward system, which runs counter to the existing literature. Following Simons’ (1995) thinking, one would expect the opposite: for the adoption of reward systems to be associated with diagnostic use. Diagnostic use is recognised as a mechanistic tool that enables operation at a distance (Simons, 1995). Since managers are given specific metrics on which they are evaluated, there is limited need for intervention by top managers. From 1995 onwards, the approach to reward systems has evolved being associated with multiple subjective measures in a compensation formula (Ittner, Larcker, & Meyer, 2003) and being complemented by dialogue among executives and managers (Kaplan & Norton, 1996). Unlike the features associated with a diagnostic control system (Simons, 1995), our results reveal the difficulties of setting incentives based on a Diag-BSC, since the measures included in the corporate report – though valuable for monitoring the overall business – are too generic to be used for setting individual objectives. This can be explained in terms of both the level of BSC dissemination within the organisation, and the purpose for which the BSC is used, which differ for the two types. First of all, the absence of cascading in a Diag-BSC means that the scorecard is not disseminated across the organisation; rather, it is a tool in the hands of senior managers. Conversely, an Int-BSC is widely disseminated throughout the organisation, since it adopts a cascading approach. The dissemination of an Int-BSC across multiple hierarchical levels makes it easier to set target for managers, because the measures are closer to the managerial activities. This is instead more difficult with a single corporate report, which includes measures that are relevant, but not readily traceable to the contributions of individual managers. Second, the purpose of use is different. A Diag-BSC represents a tool for making decisions and monitoring the overall conduct of the business – what Malmi (2001) terms an information purposes role. Given this purpose, there is no need to set incentives based on BSC measures. On the other hand, an Int-BSC, with its reports spanning all levels of the organisation, is used not only for decision making, but also to motivate managers: what Malmi (2001) terms the management-by-objective-function of the BSC. “Hard” contingency testing could prove particularly fruitful for verifying this association between the reward system and the interactive style of use.
The investigation of the specific relationships between design characteristics and PMS use indicates the existence of a reciprocal interaction between the “what” and the “how”. The misalignment problems that emerged in the SEMIC case give evidence that design and use are both equally relevant for a PMS to work. Though some previous contributions (e.g. Ferreira & Otley, 2009b) have found that style of use plays a prominent role in shaping PMS appropriateness, we did not observe this to be the case. Any changes to either the design attributes or style of use must be aligned with each other.
From a practical point of view, the misalignments emerged from the SEMIC case alert managers to ensure coherence between PMS design and use to let the system work properly, as evinced by the misalignments in the SEMIC case. Furthermore, we provide indications on how to design a BSC that encourages an interactive use. This aspect is particularly useful if managers want to enhance dialogue, learning and idea creation (Burchell, Clubb, Hopwood, & Hughes, 1980) centring around performance measures.
Finally, we point out the limitations of our case studies, and suggest some avenues for future research. First, the study suffers from the constraints inherent in the case-study methodology. The findings are theoretically grounded and internally valid, but they cannot be considered generalisable elsewhere (Yin, 2003). Second, the conceptualisation of use is based on Simons’ (1995) framework of diagnostic versus interactive. Other distinctions could have been adopted, such as Malmi’s (2001) differentiation between information purposes and management-by-objective use. However, the discrimination between diagnostic and interactive is based on the intensity of the communication process associated with the PMS, which has been recognised as relevant for analysing PMS use (Abernethy et al., 2010 and Hall, 2010).
The study points to some avenues for future research. First of all, a quantitative methodological approach could be used to test the generalisability of our findings on a wider sample. Second, it might be interesting to further explore the association we found between an interactive approach and links to the reward system, while a diagnostic use appears to lack such a link. Here, we have ascribed this result to the level of BSC dissemination and the purpose of BSC use; but additional research might detect other contingencies, or confirm our results by corroborating the same association.
