This article presents the development of a conceptual framework which aims to assess Decision Making Units (DMUs) from multiple perspectives. The proposed conceptual framework combines the Balanced Scorecard (BSC) method with the non-parametric technique known as Data Envelopment Analysis (DEA) by using various interconnected models which try to encapsulate four perspectives of performance (financial, customers, internal processes, learning and growth). The practical relevance of the conceptual model has been tested by using it to assess the performance of DMUs in a multinational company which operates in two business areas. Various models were developed with the collaboration of the directors of the company in order to conceive an appropriate and consensual framework, which may provide useful information for the company. The application of the conceptual framework provides structured information regarding the performance of each DMU (from multiple perspectives) and ways to improve it. By integrating the BSC and the DEA approaches this research helps to identify where there is room for improving organisational performance and points out opportunities for reciprocal learning between DMUs. In doing so, this article provides a set of recommendations relating to the successful application of DEA and its integration with the BSC, in order to promote a continuous learning process and to bring about improvements in performance.
In a competitive environment, characterised by the scarcity of resources, performance measurement and management assumes a crucial role. Data Envelopment Analysis (DEA) is a non-parametric technique for evaluating the performance of Decision Making Units (DMUs). Using a production metaphor, this technique, originally proposed by Charnes et al. [1], evaluates the efficiency of DMUs in converting multiple inputs into multiple outputs. Since this seminal paper, we have seen numerous theoretical developments of the DEA methodology [2]. Furthermore, we have also seen the widespread application of DEA in several contexts, such as health care, education, manufacturing, retailing, banking, etc. In recent years, we have also witnessed the development of literature relating to the need to move beyond financial measures of performance [3] and several sophisticated systems for performance assessment have been proposed. The Balanced Scorecard (BSC), developed by Kaplan and Norton [4], is one of the best-known of these performance assessment frameworks. Developed from the strategy of the organisation, this framework includes indicators related to four perspectives: financial, customers, internal processes, learning and growth.
Despite the popularity of the DEA and the BSC approaches, there have been very few studies that have explored their integration for enhanced performance assessment. This is the objective of this article. In line with what has been suggested by several authors (for example, [5], [6] and [7]), the main purpose of this research project is to explore the usefulness of Operational Research techniques (in particular, the DEA method) in real operational contexts and to put forward some recommendations regarding its successful application in practice. With this purpose in mind, and using a case study from a multinational company operating in the area of vertical transportation, we have developed four interconnected DEA models, one for each of the perspectives of the BSC. The results from these models were then analysed and discussed with the General and Regional directors of the company in Portugal in order to gain insights for performance improvement. The framework we have developed and the results it has produced suggest that moving away from a unique, all embracing DEA model, towards several complementary DEA models can be advantageous for performance measurement and performance improvement. By using several complementary models, the multidimensional nature of performance and the need to answer to the interests of multiple stakeholders is emphasised. Furthermore, the use of several complementary models offers richer information for the DMUs, because it highlights the weakest and strongest dimensions of performance and identifies relevant benchmarks for learning in each of the dimensions, acknowledging that some DMUs might be regarded as best practice in some dimensions but not in others.
We have structured the remainder of this paper into three sections. Section 2 discusses the previous studies that have combined the use of DEA with the BSC and highlights the main contribution of this article. Section 3 details the empirical study and discusses the main results. In particular, in this section, we discuss the development of the BSC and the DEA models to capture each of the performance dimensions and the use of the results to gain insights for performance improvement. Section 4 concludes and offers suggestions for future research.
This paper presented an integration of two of the most popular methods used for organisational performance evaluation: the DEA and the BSC. Lewin and Minton [49] reviewed the extent to which the components of a contingent behavioural theory of organisational effectiveness can incorporate the paradoxes and tradeoffs inherent in real life organisations. In their paper, they emphasise that there cannot be a unique and universal model of organisational effectiveness. Wholey [50] has also emphasised that performance is socially constructed and means different things for different stakeholders. In this respect, we argue that moving away from a unique all embracing DEA model towards multiple complementary models is advantageous, leading to enhanced performance assessment. In evaluating the performance of decision making units, it is essential to make explicit from whose perspective is the evaluation [15] and [39]. Answering to the interests of one stakeholder may conflict with answering the interests of other stakeholders. Using a unique all embracing DEA model hides the complexity involved in performance assessment and may fail to identify dimensions of performance that require attention. Furthermore, we argue that it is advantageous to identify the most appropriate benchmarks for each one of the performance dimensions. For example, the most appropriate benchmarks from a financial perspective may not be the most appropriate ones from an internal processes perspective.
In a case study of a multinational company operating in the business of vertical transportation, we developed four DEA models, one for each one of the BSC perspectives. The fact that our results do not show a high correlation between the scores from the four perspectives, also confirms that, in this context, it is advantageous to move away from an all embracing DEA model towards several complementary models capturing different dimensions of performance. Our results have also shown that an in depth analysis of the weights attached to the same variable under different perspectives can offer insightful information for management regarding production tradeoffs.
However, the results from this case study have to be interpreted with caution, because we have used data comprising a single segment of a company for one year. This poses some limitations, as it is not possible to statistically generalise from these results.
It is important to emphasise, however, that the objective of this study was not to ensure statistical generalisation of the results but rather perform an in-depth formative evaluation, focusing on disaggregated production processes, as an attempt to open the input–output transformation box and identify some of the structures and mechanisms behind successful practice. Despite its limitations, the case study shows the potential for DEA to contribute to process improvement interventions and it is our belief that the insights derived from it can inform implementations in other contexts.
In future research studies, it would be interesting to perform dynamic analyses in different contexts in order to better understand the relationships between the different dimensions of performance. In particular, it would be important to test the cause and effect relationships hypothesised by the BSC advocates and to explore whether there is a temporal gap between the impacts of performance of the leading perspectives on the lagging ones. In modelling these relationships, the dynamic DEA model, initially proposed by Färe and Grosskopf [51] and recently extended by Tone and Tsutsui [52], can be very useful.
