بررسی تاثیر اندازه گیری عملکرد ارزیابی شده و تجزیه و تحلیل استراتژیک بر ارزیابی ریسک حسابرسان و مدل های ذهنی

As the audit environment becomes more demanding and complex, so does the set of analytical tools available to an auditor. The purpose of this paper is to examine the effect of two complex audit technologies commonly used by auditors, benchmarking of performance measures and strategic analysis, on the risk judgments of auditors carrying out the initial planning of an audit. We conduct an experiment that utilizes a Balanced Scorecard for organizing and evaluating analytical evidence about the performance of business units within a large client. Our first principal finding is that external benchmarking can cause an auditor to focus on performance measures that are unique to a business unit and disregard performance measures that are common to multiple business units but not benchmarked. However, our second finding is that an in-depth strategic analysis completed prior to assessing a client’s business risk or risk of material misstatement allows an auditor to incorporate more information from performance measures in risk assessments regardless of whether the performance measures are benchmarked. Strategic analysis facilitates a more balanced and accurate assessment of the risks across the business units being evaluated. We also provide evidence that the latter result occurs because in-depth strategic analysis allows auditors to develop a more complete mental model of a client, which has been a long time belief of advocates of business risk audit methodologies and consistent with current and emerging auditing standards on risk assessment.

The analysis of a client’s strategy and related business risks is an integral part of the auditor’s process for evaluating the risk of material misstatements in a client’s financial report.3 In general, strategic analysis used in an audit includes assessing the competitive position (Porter, 1985) adopted by the firm, identifying its critical success factors, and evaluating obstacles that might cause a chosen strategy to be less successful than desired (e.g., Knechel, 2007). While the strategic management literature employs such analysis to identify investment or operating opportunities (Porter, 1979 and Porter, 1985), auditors view threats to client business success as a source of audit risk. More specifically, the idea is to link the auditor’s understanding of a client’s business environment and strategic plans to the analysis of audit risk. Strategic analysis may be conducted at the corporate level and supplemented with an analysis of individual strategic business units (Morrison & Roth, 1992). Consequently, an auditor will often assess the relative business risks associated with a strategic business unit (SBU) as a precursor to assessing audit risk and allocating audit effort.4 One of the most frequently used tools in strategic analysis is the evaluation of performance data at either the corporate or business unit level (Kaplan & Norton, 1996). Management accountants emphasize that the use of performance measures at the unit level should reflect the individual unit’s strategy for success (Kaplan and Norton, 1996, Kaplan and Norton, 2001 and Simons, 2000). In the case of multiple business units, the same performance measures may be used for different units or they may be unique to a specific unit. Since auditors are required to use analytical procedures to assess the risk of material misstatement (ISA, 2003 and SAS, 2008), performance measures can be used by an auditor to highlight situations where problems may exist in management’s strategy that can be a source of audit risk (Kinney and McDaniel, 1996, McDaniel and Kinney, 1995 and Waddington et al., 2001). Unit-level performance measures can also be used to judge audit risks within an organization in more detail (Mediori and Steeple, 2000, Thomas et al., 1999 and Venkatraman, 1989). Further, organizations often use benchmarking to assist in the evaluation of their results so auditors can use the same benchmarks to facilitate their evaluation of audit risk (Knechel et al., 2007 and Thompson and Strickland, 1998).5 Finally, if benchmarks are used by a client, they are potentially most informative when combined with a top-down view of a client’s strategic business plan (Murray, Zimmermann, & Flaherty, 1997). The purpose of this study is to examine the joint effect of strategic analysis and benchmarking of performance measures on auditor judgment in assessing audit risk. First, we hypothesize that benchmarking can help the auditor to incorporate the implications of a broad set of performance measures when making judgments about audit risk. Second, we hypothesize that an auditor’s use of strategic analysis influences the manner in which auditors evaluate performance measures, and more extensive strategic analysis allows auditors to incorporate more information from a broader set of performance measures into their risk assessments. We test our hypotheses with an experiment involving 87 audit seniors from a Big 4 audit firm in Canada. During the experiment, participants were asked to rate the performance and risk level of two business units in the same organization. To test our hypotheses, we manipulated (1) the presence or absence of benchmarks for different types of performance measures (unit-specific or common across units) and (2) the extent of strategic analysis available to the auditor (in-depth vs. superficial). Our results indicate that auditors can incorporate more information into their risk assessments when given benchmarked performance measures that are common to multiple business units, as compared to when only unit-specific performance measures are benchmarked. With respect to our second hypothesis, we find that risk assessments incorporate a broader set of information available to the auditor when auditors have an extensive strategic analysis available to them, regardless of the type of performance measures and benchmarks that are available. Finally, we report supplemental results based the theory of mental models (e.g. Legrenzi, Girotto, & Johnson-Laird, 1993) indicating that auditors receiving an in-depth strategic analysis develop mental models that are more complete than those based on a superficial strategic analysis. These results support our assertion that differential cognitive processing of benchmarked performance is affected by the depth of strategic analysis. The remainder of this paper proceeds as follows. First, we develop the theoretical basis for our hypotheses about the effects of benchmarks and types of performance measures on auditor risk judgments, as well as the ameliorating effects of in-depth strategic analysis. Second, we outline the experimental design and related procedures. Third, we present the results of our study. We conclude with a discussion of the study’s limitations and contributions.

In this paper, we examine how characteristics of the client performance measurement system and auditor strategic analysis interact when auditors are making risk assessments. We hypothesize conditions under which an auditor’s risk assessments may be improved by performance benchmarks and strategic analysis. For our purposes, we define “better” judgment as incorporating more decision-relevant information from the set of performance measures available.28 Our results indicate that auditors incorporate the information from a broader set of performance measures in their assessment of client business risk and the risk of material misstatement when benchmark data is available for performance measures that are common to multiple business units. On the other hand, benchmarking of unit-specific performance measures causes auditors to overlook or downplay the information contained in other performance measures that are not benchmarked but which are common to multiple business units. Having established the potential effect of benchmarks in a setting with a superficial strategic analysis, we then observe that auditors using an in-depth strategic analysis of a client incorporate the risk implications of a broader set of performance measures in their judgments, regardless of the availability of benchmarks. Supplemental analysis suggests that this result is due to the development of a richer mental model by auditors who have an in-depth strategic analysis available when evaluating unit risk. Our results also indicate that differences in an auditor’s perception of a unit’s relative performance affect the assessment of the risk for a unit. The limitations of this study include the use of auditors with approximately two years of experience who are all employees of the same firm; however, these are the staff who normally carry out this task; use of a single case with less information than would be available in practice, however with substantially more information than is usually provided in an experimental setting; use of a generic version of the firm’s audit approach; and the administration of the experiment on paper rather than via the firm’s automated audit software (the latter two due to the firm’s concerns about proprietary systems). These limitations are offset by having access to a relatively homogenous set of auditors whose training and practice experience provided them with extensive use of strategic analysis tools and whose audit approach routinely employed information from the client’s performance measurement systems. Extensive pilot testing supports our contention that the participants who had the same information available could develop a rich strategic analysis if not presented to them in the case materials, thus allowing the research to focus directly on the depth of analysis issue. There are at least two important insights gained from this study. First, the effect of benchmarks on risk evaluation is conditional on whether the performance measure being benchmarked is unique to a specific unit. The fact that the benchmarking of unit-specific measures results in auditors processing less information in their risk assessments may be due to the cognitive difficulty of processing unit-specific benchmarks. Since auditing standards strongly encourage auditors to consider analytical evidence that can be benchmarked against external independent sources, the presence of benchmarks for unit-specific measures essentially forces auditors to put much cognitive effort into evaluating those performance measures. However, since unit-specific measures are not readily alignable with other units’ performance measures, the effort used on evaluating one division does not readily transfer to the evaluation of another division. Therefore, there is little cognitive synergy across units as each unit must be evaluated somewhat independently of the other given that unit-specific benchmarks require more complex information processing. As partial sets of externally benchmarked performance measures are likely to be found in performance measurement systems or an auditor’s proprietary data bases, these situations are likely to be encountered by auditors on a regular basis. From a conceptual perspective, this study provides preliminary insight into how an auditor maps performance to risk. The mapping does not seem to be simple or isomorphic, otherwise we would expect to see a common measures bias story (cf. Lipe & Salterio, 2000) emerge from our analysis as hypothesized in H1a. Rather, the complexities of audit standards, performance measure benchmarked type and auditor professional skepticism comingle to create a complex, potentially polymorphic, mapping of information contained in performance measures to an auditor’s risk assessments. Second, we find that auditors provided with in-depth strategic analysis were more likely to reach the normatively appropriate conclusion that the two units are equally risky, regardless of the presence or absence of benchmarks for subsets of performance measures. This result suggests that the cognitive effort needed to process benchmarked unit-specific performance measures may be reduced when an auditor starts the task of risk assessment with a well-developed strategic analysis already in place. Our supplemental analysis suggests that the improved information processing is due to auditors developing a more complex mental model of the client’s environment which facilitates improved assessment of the client’s business risks and risk of material misstatement. The finding that in-depth strategic analysis facilitates more accurate judgments about risk provides support for international professional standards on risk assessment (ISA 315, effective for audits in 2005), as well as the recently proposed standards of the PCAOB (see PCAOB Docket 2008-06 released October 21, 2008, Proposed Auditing Standards — Auditor’s Assessment of and Response to Risk), both of which focus on greater understanding of the client and its industry and an integration of that knowledge into risk assessments for the audit.