تنظیمات ذهنی برای اندازه گیری عملکرد عینی: تاثیر عملکرد قبلی

This field study examines whether and how supervisors’ subjective adjustments to objective performance measures are influenced by their prior subjective evaluations of employees. Evaluations were determined entirely subjectively in the sample internal audit organization in 2005. In 2006, the organization introduced a pay-for-performance incentive plan that established four objective measures of audit manager performance. Then, knowing the challenges of objectively measuring manager performance, the organization gave supervisors the discretion, mandate, and training to subjectively adjust each of the objective measures when performance as indicated on the individual measures misrepresented managers’ true performance. Using prior-year subjectively measured performance to proxy for current-year expected performance, empirical evidence documents that upward adjustments are more likely to be made to unexpectedly low individual measures the more supervisors perceive deficiencies in those objective measures. This indicates that supervisors made adjustments to correct deficiencies in the measures (as the organization intended). Independent of this interaction effect, however, unexpectedly low current-year objectively-measured performances are also more likely to be adjusted upward, which indicates supervisors also made current performance consistent with prior performance for reasons other than to improve individual objective measurement. Some of these other reasons are explored. The study highlights how the impact of the implementation of a new performance measurement system depends on the past.

This study examines whether and how supervisors’ subjective adjustments to objective performance measures are influenced by their prior subjective evaluations of employees.1 A large internal audit organization provides the sample setting. In 2005, supervisors evaluated audit managers entirely subjectively. In 2006, the organization introduced a new performance measurement system that tied audit manager incentives to four objective performance measures. Then, knowing the challenges of objectively measuring manager performance, the organization gave supervisors the discretion to subjectively adjust each of the individual objective measures when performance, as indicated on the measures, misrepresented managers’ true performance. To understand how supervisor behavior is influenced by a transition to a new performance measurement system, this study integrates behavioral theory on “assimilation effects” with economic theory on performance measurement. Assimilation effects would occur in my setting if supervisors use adjustments to make current-year objective performance consistent with their prior-year subjective evaluations. If supervisors make current performance consistent with prior performance when they perceive that the current objective performance is deficiently measured, then they likely make adjustments consistent with their mandate of improving objective performance measurement. In contrast, if supervisors make current performance consistent with prior performance for other reasons, then they likely use subjective adjustments to pursue their own goals. Distinguishing the reasons adjustments are made is important because the success of the implementation of new measurement systems is likely affected by whether supervisors make adjustments in accordance with their mandate. To study this issue, I combine proprietary performance evaluation data with survey measures of supervisor perceptions of deficiencies in the objective performance measures. I find that supervisors are more likely to raise unexpectedly low current-year objectively-measured performances the more they perceive the measures of those dimensions of performance are noisy and incomplete. Independent of this interaction effect, however, unexpectedly low current-year objectively-measured performances are also more likely to be raised, which indicates supervisors also made current performance consistent with prior performance for reasons other than to improve individual objective measurement. Instead of lowering unexpectedly high performances that are deficiently measured, supervisors appear to use downward adjustments to encourage the departure of certain managers and avoid using downward adjustments to preclude negative consequences for managers and themselves. Overall, evidence is consistent with supervisors using their discretion to both improve objective measurement and to pursue their own goals. This study contributes to prior literature in several ways. First, it highlights how the effect of the implementation of a new performance measurement system depends on the history of the prior system. Many organizations frequently change their performance measurement systems, yet relatively little is known about how such performance measurement system implementation affects supervisor behavior. The organization in Ittner, Larcker, and Meyer (2003) changed its system three times from 1993 to 1998. The evidence put forth in that paper suggests that supervisors used their discretion to prioritize financial performance—as had been done in the past—and thereby removed the balance in the newly introduced balanced scorecard. It was not exclusively focused, however, on whether and how the transition to a new performance measurement system affects supervisor behavior, as is the current study. Second, this study complements a recent study by Bol and Smith (2011). Whereas it examines how objective performance on one task influences a subsequent subjective assessment on another task in an experimental setting, the current study considers how prior subjective assessments influence later assessments of objective measures in a field setting. The distinction between the reliance on a prior subjective versus objective measure on a current evaluation is an important one because a number of factors related to objective and subjective measure differences may influence whether and how supervisors make current performance consistent with prior performance.2 In addition, although Bol and Smith (2011) examine the effect of a noisy measure on that process, I examine how several measure properties, of which noise is one, influence the process. Third, this study extends recent work by Höppe and F. Moers. (2011). It shows how performance measure noise explains cross-sectional variation in the design of incentive contracts and, in particular, when different types of subjectivity might be used for CEOs. My study, in contrast, shows how performance measure properties, including noise, affect the application of subjectivity within a certain incentive contract design for middle-level managers. Although some other studies also examine the application of subjectivity (e.g., Gibbs, Merchant, Van der Stede, & Vargus, 2004), they do not consider how the transition to a new measurement system affects supervisor behavior. Finally, prior studies in the management literature on performance evaluation have investigated various reasons why supervisors make current performance consistent with prior performance, but by integrating economic theory on performance measurement with behavioral theory, this is the first study to consider that assimilation effects may result from supervisors correcting measure deficiencies. The remainder of the paper is organized as follows: the research setting section describes the research setting, the theory section formalizes the research hypotheses, the variable measurement and empirical specification section describes the measurement of the variables and the empirical specifications used to test the hypotheses, the results section presents the results, and the summary and conclusion section provides the results’ limitations and implications.

My tests reveal a number of findings. Most adjustments (95%) are upward. Supervisors raise current, unexpectedly low performance so that it is consistent with prior performance when they perceive the measure of that performance is incomplete and noisy, consistent with their mandate of improving objective measurement. Supervisors also raise current, unexpectedly low performance so that it is consistent with prior performance for other reasons, and even apparently holistically (at the manager level). Evidence documents that supervisors make downward adjustments to encourage some employees to leave the organization, and avoid downward adjustments to preclude negative consequences for managers and themselves. Overall, despite the organization’s best attempts to focus all supervisors on the same purpose of improving individual objective measurement, evidence is consistent with supervisors using their discretion over subjective adjustments in a variety of ways. The apparent rogue supervisor behavior shown in this study has also existed in other studies of subjectivity (e.g., Ittner et al., 2003). This paper has a number of limitations. For example, the relative importance of adjustments for measurement properties vis-à-vis “other” reasons remains unclear. Moreover, I only examine a limited number of possibilities, but there are myriad ways in which supervisors could have used adjustments. In addition, given the manner in which the other reasons were measured, they may be related to adjustments by construction only. I also did not measure the performance measures’ “true” deficiencies. Instead, I asked survey participants to judge these deficiencies, and because these judgments were subjective, they likely contained some noise and/or bias. However, I expected supervisors to make subjective adjustments, not based on the true extent of the deficiencies, which they do not know, but based on their perceptions of measure deficiencies. Consistent with prior subjectivity studies, my results suggest that there are many economic and psychological variables affecting subjective adjustments. Indeed, there are limitations in the theory that explains the uses of subjectivity, due in part to the lack of empirical data on this increasingly important topic (Gibbs et al., 2004 and Prendergast, 1999). I also assume that unexpected performance is indicated by deviations between prior year subjectively-measured performance and current-year objectively-measured performance, when “true” manager performance may have actually varied. I mitigate this concern, however, with evidence showing that supervisors’ beliefs about manager performance were remarkably consistent year-on-year. The main test for downward adjustments could not support or refute the theory. A data set with a forced ranking system in place (where upward adjustments to one manager/measure automatically implies a downward adjustment to another), for example, would, provide a more ideal test. Finally, this study is subject to the usual generalizability caveats of a field study, and thus may generalize to similar settings, such as those with low control spans and complex work. This paper has several implications for future research. It would be fruitful to examine how various combinations of subjectivity influence incentive system optimality. Identifying how the benefits and/or costs of subjectivity are impacted by various monitoring systems would also be interesting. I leave the identification of these issues to future research (Ittner & Larcker, 1998).