سنجش و اصول الگوبرداری : حمایت از سازمان ها در توسعه و رشد آنها با استفاده از داشبوردها

Growth and stage models often lack a sound empirical and theoretical base and do not provide any help for organizations to improve. Measuring and benchmarking (M&B) is necessary for understanding an organization's position and identifying growth opportunities. Yet M&B methods are often not based on generalizations of practice and measure only what is directly visible. They are missing relevant elements that can help further development. In this paper, we propose a multi-level measurement framework utilizing a mix of measurement methods to look deep inside organizations. Whereas benchmarking is often based on a single number, deep insight is given by showing the performance in a broad range of areas and views using a dashboard. Guidance for improvement is created by identifying those elements that need improvements. The illustration of the framework in a case study shows that the process of measuring deep inside organizations might be more important than the actual outcomes and that per area different maturity levels might be possible. We provide seven principles that can serve as a foundation for developing M&B and stage models.

Measurement and benchmarking (M&B) of public organizations is becoming increasingly important for governments and organizations to help them in their development and to take advantage of the newest developments. Benchmarking is the measurement of some elements and the comparison of the outcomes to a certain norm, the benchmark. Whereas the focus of benchmarking is on mutual learning, practice often shows a narrow focus on numbers (Bannister, 2007), most models are developed for certain situations (Coursey & Norris, 2008), and success has been limited so far (Ojo, Janowski, & Estevez, 2011). Measuring is aimed at determining the performance based on some kind of criterion, whereas benchmarking is the activity to compare the resulting scores with some kind of norm. Norms are often derived based on measuring results of other organizations. This type of comparison requires that similar measures can be used among organizations. The results of M&B activities should result in organizational improvement and stimulate organizational learning. Yet many M&B methods do not provide any learning and there is much criticism (see for example Bannister, 2007, Janssen et al., 2004 and Peters et al., 2004). Despite the criticisms, limited attention has been given to develop foundations and guidance for developing improved methods. Stages-of-growth models are often used to represent the current status e-government (Peters et al., 2004). Often stages are modeled with sequential steps showing the growth, whereas many models are incongruent with each other (Lee, 2010 and Siau and Long, 2005). The position within a certain stage is ideally determined using measures and the benchmarking norms are determined by measuring other organizations. Whereas a sound measurement model might be thought as the basis of stage models, often this is not the case. Stage-of-growth models are often based on intuitive appealing models without providing any guidance to determine in which stage an organization is. Often the focus of measurement is on a generic level at the expense of detailed insights (Bannister, 2007 and Kunstelj and Vintar, 2004). This difficulty might result in the adverse effects that benchmarks might have limited practical meaning, but might have a huge impact on political decision-making (Bannister, 2007). The measurement is mainly based on the outcome of the past events, is often rather abstract and hardly takes the size, scope, and complexity of the government organizations into account (Bannister, 2007 and Gupta and Jana, 2003). This makes it hard to understand the position of an organization and to identify opportunities for improvement. In conclusion, M&B and stages-of-growth models provide hardly any support for organizational development. They can only be used as a start for organizations by giving a global idea about the current position, but leave a void which actually can and should be done and what specific areas are those that need to be improved. The goal of this paper is to develop a M&B method that provides organizations guidance in their development to a higher maturity. Current measurement models provide either high-level views or are focused on the particular aspects and provide limited support for organizational development. This paper extends the framework of the paper of Maheshwari, Janssen, and Veenstra (2011) published at the ICEGOV2011 conference by including analyses of practical and theoretical challenges and introducing seven principles for architecting stages-of-growth and M&B models. In Section 2 literature is reviewed and existing M&B models are evaluated. In Section 3 the research method is presented. The multi-level measurement framework and constructs are presented in Section 4. In Section 5, the framework is illustrated in a case study at the Inland Revenue (IR) Karachi, Pakistan. Finally, in Section 6 we propose foundations for M&B and draw conclusions and recommendations.

The research in this paper aimed at developing a M&B method that provides organizations guidance in their development towards higher maturity. M&B should not be viewed as an end, instead it should be considered as a means to improve organizations. The literature survey shows that many M&B and stage models copy each other's measurement methods, but do not address the limitations and challenges of M&B methods; they merely apply the same concepts to other areas. This might be a quick win, but does not advance our knowledge in the field of M&B and often results in yet another model. Derived from the criticisms as found in the literature eight theoretical and practical challenges of M&B and stage models were identified. The four theoretical challenges include 1) a lack of theoretical and empirical bases, 2) use of shallow and incomplete measurement and limited understanding, 3) focus on limited number of aspects, and 4) no improvement support. The four practical challenges include 5) utilization of M&B resources and costs, 6) measurement of proxies, resulting in ambiguous interpretation, 7) ambiguous performance metrics, and 8) not being able to deal with differences in organizations, processes, operations, cultures etc. To overcome these challenges we developed a measurement approach consisting of a multi-level measurement approach, utilizing measurement constructs and variables derived from literature and visualizing the results using dashboards. Data collection is based on multiple data collection method instead of relying on a single method. This model uses areas and views to gain a more in-depth overview. The use of levels, areas and views in combination with multiple measurement methods helps to measure deep inside the organizations. Instead of having a single outcome, it is possible to see the performance per area or per view. In some areas or by taking a certain view, an organization might have a low score, whereas in other areas or views it might perform well. Organizations can benefit from the M&B method not only by knowing their position on the benchmark in the various areas they are involved in, but also by having specific guidance for improvement they can learn from the actions and interventions taken by others and the impact of these interventions. Benchmarking is more than comparing with a certain number. Benchmarking is about identifying and sharing practices with each other. Developing measurement and stage models might be simple as demonstrated by the huge amount of models found, however, developing empirical and theoretical sound models is challenging. Based on the evaluation in the case study we abstracted seven foundational principles underlying our M&B method. Table 4 shows which principles are used to address which challenge and the resulting benefits of its use. Each principle contributes to the B&M and has different implications. These foundational principles help to derive stage models, measurement, and benchmarking models having a better foundation. In further research, these principles can be used to derive new or improved stage and M&B models and the principles can be further refined and extended. In particular, dashboards are hardly investigated by researchers and deserve ample research attention. For example, it is still unclear what types of dashboards are most suitable, which views should be included and how different stakeholders can make use of dashboards. In addition, the principles for visualizations of the M&B can be further refined and extended.The use of the foundations might require the making of trade-offs to focus on certain aspects. For example, the use of multiple methods can increase the validity, but requires more resources than a single method. Policy-makers should make the trade-off between using more resources, overcoming the disadvantages and gaining the advantages. The framework in this paper helps to make such decisions, however, it does not offer support for making these decisions. In further research decision support can be created to use the foundations and make practical trade-offs. M&B is a complex phenomenon, which encompasses a broad range of theoretical and practical challenges. Using the principles can help to have better theoretical foundations, which avoids the development of yet another stage or measurement model, but the principles should not be viewed as an end.