Description
This field study examines whether and how supervisors’ subjective adjustments to objective
performance measures are influenced by their prior subjective evaluations of employees.
Evaluations were determined entirely subjectively in the sample internal audit
organization in 2005. In 2006, the organization introduced a pay-for-performance incentive
plan that established four objective measures of audit manager performance. Then,
knowing the challenges of objectively measuring manager performance, the organization
gave supervisors the discretion, mandate, and training to subjectively adjust each of the
objective measures when performance as indicated on the individual measures misrepresented
managers’ true performance.
Using prior-year subjectively measured performance to proxy for current-year expected
performance, empirical evidence documents that upward adjustments are more likely to
be made to unexpectedly low individual measures the more supervisors perceive deficiencies
in those objective measures.
Subjective adjustments to objective performance measures:
The in?uence of prior performance
Alexander Woods
?
The College of William and Mary, P.O. Box 8795, Mason School of Business, Williamsburg, VA 23187-8795, United States
a b s t r a c t
This ?eld study examines whether and how supervisors’ subjective adjustments to objec-
tive performance measures are in?uenced by their prior subjective evaluations of employ-
ees. Evaluations were determined entirely subjectively in the sample internal audit
organization in 2005. In 2006, the organization introduced a pay-for-performance incen-
tive plan that established four objective measures of audit manager performance. Then,
knowing the challenges of objectively measuring manager performance, the organization
gave supervisors the discretion, mandate, and training to subjectively adjust each of the
objective measures when performance as indicated on the individual measures misrepre-
sented managers’ true performance.
Using prior-year subjectively measured performance to proxy for current-year expected
performance, empirical evidence documents that upward adjustments are more likely to
be made to unexpectedly low individual measures the more supervisors perceive de?cien-
cies in those objective measures. This indicates that supervisors made adjustments to cor-
rect de?ciencies in the measures (as the organization intended). Independent of this
interaction effect, however, unexpectedly low current-year objectively-measured perfor-
mances are also more likely to be adjusted upward, which indicates supervisors also made
current performance consistent with prior performance for reasons other than to improve
individual objective measurement. Some of these other reasons are explored. The study
highlights how the impact of the implementation of a new performance measurement sys-
tem depends on the past.
Ó 2012 Elsevier Ltd. All rights reserved.
Introduction
This study examines whether and howsupervisors’ sub-
jective adjustments to objective performance measures are
in?uenced by their prior subjective evaluations of employ-
ees.
1
A large internal audit organization provides the sample
setting. In 2005, supervisors evaluated audit managers en-
tirely subjectively. In 2006, the organization introduced a
new performance measurement system that tied audit man-
ager incentives to four objective performance measures.
Then, knowing the challenges of objectively measuring man-
ager performance, the organization gave supervisors the dis-
cretion to subjectively adjust each of the individual objective
measures when performance, as indicated on the measures,
misrepresented managers’ true performance.
To understand how supervisor behavior is in?uenced by
a transition to a new performance measurement system,
this study integrates behavioral theory on ‘‘assimilation ef-
fects’’ with economic theory on performance measure-
ment. Assimilation effects would occur in my setting if
supervisors use adjustments to make current-year objec-
tive performance consistent with their prior-year subjec-
tive evaluations. If supervisors make current performance
0361-3682/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.aos.2012.06.001
?
Tel.: +757 221 2967.
E-mail address: [email protected]
1
An objective measure exists as a quantity in and of itself; in contrast,
subjective measurements are based on attitudes, beliefs, and perceptions.
These de?nitions are consistent with that of prior literature (Rajan &
Reichelstein, 2009).
Accounting, Organizations and Society 37 (2012) 403–425
Contents lists available at SciVerse ScienceDirect
Accounting, Organizations and Society
j our nal homepage: www. el sevi er. com/ l ocat e/ aos
consistent with prior performance when they perceive that
the current objective performance is de?ciently measured,
then they likely make adjustments consistent with their
mandate of improving objective performance measure-
ment. In contrast, if supervisors make current performance
consistent with prior performance for other reasons, then
they likely use subjective adjustments to pursue their
own goals. Distinguishing the reasons adjustments are
made is important because the success of the implementa-
tion of new measurement systems is likely affected by
whether supervisors make adjustments in accordance with
their mandate.
To study this issue, I combine proprietary performance
evaluation data with survey measures of supervisor per-
ceptions of de?ciencies in the objective performance
measures. I ?nd that supervisors are more likely to raise
unexpectedly low current-year objectively-measured per-
formances the more they perceive the measures of those
dimensions of performance are noisy and incomplete.
Independent of this interaction effect, however, unex-
pectedly low current-year objectively-measured perfor-
mances are also more likely to be raised, which
indicates supervisors also made current performance con-
sistent with prior performance for reasons other than to
improve individual objective measurement. Instead of
lowering unexpectedly high performances that are de?-
ciently measured, supervisors appear to use downward
adjustments to encourage the departure of certain man-
agers and avoid using downward adjustments to preclude
negative consequences for managers and themselves.
Overall, evidence is consistent with supervisors using
their discretion to both improve objective measurement
and to pursue their own goals.
This study contributes to prior literature in several
ways. First, it highlights how the effect of the implementa-
tion of a new performance measurement system depends
on the history of the prior system. Many organizations fre-
quently change their performance measurement systems,
yet relatively little is known about how such performance
measurement system implementation affects supervisor
behavior. The organization in Ittner, Larcker, and Meyer
(2003) changed its system three times from 1993 to
1998. The evidence put forth in that paper suggests that
supervisors used their discretion to prioritize ?nancial per-
formance—as had been done in the past—and thereby re-
moved the balance in the newly introduced balanced
scorecard. It was not exclusively focused, however, on
whether and how the transition to a new performance
measurement system affects supervisor behavior, as is
the current study.
Second, this study complements a recent study by Bol
and Smith (2011). Whereas it examines how objective per-
formance on one task in?uences a subsequent subjective
assessment on another task in an experimental setting,
the current study considers how prior subjective assess-
ments in?uence later assessments of objective measures
in a ?eld setting. The distinction between the reliance on
a prior subjective versus objective measure on a current
evaluation is an important one because a number of factors
related to objective and subjective measure differences
may in?uence whether and how supervisors make current
performance consistent with prior performance.
2
In addi-
tion, although Bol and Smith (2011) examine the effect of
a noisy measure on that process, I examine how several
measure properties, of which noise is one, in?uence the
process.
Third, this study extends recent work by Höppe and F.
Moers. (2011). It shows how performance measure noise
explains cross-sectional variation in the design of incentive
contracts and, in particular, when different types of subjec-
tivity might be used for CEOs. My study, in contrast, shows
how performance measure properties, including noise, af-
fect the application of subjectivity within a certain incen-
tive contract design for middle-level managers. Although
some other studies also examine the application of subjec-
tivity (e.g., Gibbs, Merchant, Van der Stede, & Vargus,
2004), they do not consider how the transition to a new
measurement system affects supervisor behavior.
Finally, prior studies in the management literature on
performance evaluation have investigated various reasons
why supervisors make current performance consistent
with prior performance, but by integrating economic the-
ory on performance measurement with behavioral theory,
this is the ?rst study to consider that assimilation effects
may result from supervisors correcting measure
de?ciencies.
The remainder of the paper is organized as follows: the
research setting section describes the research setting, the
theory section formalizes the research hypotheses, the var-
iable measurement and empirical speci?cation section de-
scribes the measurement of the variables and the empirical
speci?cations used to test the hypotheses, the results sec-
tion presents the results, and the summary and conclusion
section provides the results’ limitations and implications.
Research setting
Pay-for-performance incentive plan
This paper’s research setting comprises a large internal
audit organization. Prior to 2006, the organization pro-
vided its audit managers with annual salary increases
based on seniority. A nominal bonus was also provided
based on an overall subjective evaluation.
3
As part of a
comprehensive ‘‘performance management plan’’ aimed at
recruiting, retaining, and rewarding talented human capital,
the organization introduced a pay-for-performance incen-
tive plan in 2006. Before introducing the new plan, the orga-
nization developed four objective performance measures
based on ‘‘ideal’’ properties of objective measures that corre-
spond to academic theories of performance measurement
2
Moreover, the use of the prior objective performance information was
irrelevant in the arti?cial setting they created. Conversely, using prior
performance information can be relevant, I argue, in my setting because it
provides an expectation of what current performance on the four newly-
developed objective performance measures should be, and because super-
visors’ focus should be on trying to determine when those measures
misrepresent ‘‘true’’ performance.
3
Managers were evaluated on nine subjective measures (e.g., work
effort, communication) on a scale from 1 to 9. Thus, theoretically, overall
performance could range from 9 to 81. Actual 2005 overall subjective
evaluations ranged from 66 to 80.
404 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
[e.g., within employee control (i.e., precise), aligned with
organizational objectives (i.e., congruent), etc.]. The four
measures—Planning, Program Management, Reporting, and
Professional Development—were ones the organization be-
lieved captured entirely separate dimensions of audit man-
ager performance. The organization standardized and
equally weighted the four measures, then made managers’
annual salary increase and bonus dependent on managers’
performance on them.
The Planning measure is designed to instill ownership
over future audit work. Speci?cally, this measure requires
managers to annually identify and develop promising fu-
ture audit subjects, thereby contributing to the develop-
ment of a risk-based plan focused on improving the
organization’s processes. Success on this measure requires
managers to integrate knowledge and technical pro?ciency
to best serve future customer needs.
The Program Management measure is designed to cap-
ture the bulk of audit managing work. Program manage-
ment requires audit managers to perform preliminary
research, program design, application, summarization,
and analysis work that meets speci?ed organizational stra-
tegic plans and requirements, and generally accepted
auditing standards. Managers must identify resource
requirements prior to the audit’s application phase. They
must provide technical guidance and assistance to ?eld
auditors during this phase to facilitate timely audit com-
pletion. Managers must also prepare work papers that in-
clude evidence suf?cient to support audit objectives and
conclusions without material omissions. Managers must
pro?ciently use sophisticated computer techniques to en-
hance the audit process and analyze collected audit data.
They must also inform supervisors of all critical changes
to the audit plan, such as milestone delays, audit applica-
tion problems, and customer requests.
The Reporting measure is designed to capture the end
product of an audit manager’s work. Speci?cally, the ?nal
result of an audit is a unique audit report (or other product
such as outlines, drafts, or brie?ngs) that presents audit re-
sults, identi?es causes, and recommends corrective ac-
tions. The Reporting measure requires audit managers to
write audit reports that communicate effectively, with
minimal revisions. It also requires managers to defend
impartial audit ?ndings and recommendations to organi-
zation management of?cials, while keeping customers in-
formed and considering their views and disagreements.
The Professional Development measure is designed to
ensure that audit managers retain competency in their area
of expertise. It requires audit managers to obtain contin-
uing education and professional development training.
The organization believes that acquiring and using job-spe-
ci?c knowledge drives organizational performance. Conse-
quently, managers must complete functional training and
meet pre-speci?ed education requirements and standards.
Each performance measure is ‘‘objective’’ because per-
formance as measured against and compared to the stan-
dards actually existed as quantities before performance
was adjusted. Speci?cally, performance standards for each
measure were established prior to the measurement peri-
od. Then, at the end of the measurement period, perfor-
mance was compared to the pre-established standards
and recorded on a scale from one to ?ve. For example, for
the Planning measure, managers were expected to identify
and document two audit subjects for inclusion in the orga-
nization-wide triennial audit plan. Managers who contrib-
uted two audit subjects received a ‘‘3’’ on the Planning
measure. Contributing four audit subjects earned manag-
ers a ‘‘5’’ because their performance on this measure ex-
ceeded the established standard. Likewise, contributing
fewer than two audit subjects earned managers less than
a ‘‘3.’’ Subjectivity was likely used in the determination
of performance standards, but actual performance as com-
pared to the standards was simply recorded as a quantity
before supervisors could then make their adjustments. I
deem these measures objective because they existed as
quantities (prior to supervisors adjusting them) and could
be easily veri?ed, just as objective ?nancial accounting
measures such as pro?t exist as a quantity despite layers
of subjectivity underlying them.
Theoretically, each manager would have different
weights for each measure; this organization, however, like
many others, believed the bene?ts of standardized mea-
sures and weights exceeded the costs (Arya, Glover, Mit-
tendorf, & Ye, 2005). Nevertheless, knowing the nature
and increasing complexity of internal audit work due, for
example, to legislation such as the U.S. Sarbanes–Oxley
Act of 2002 (McDonald, 2003), and knowing the nature of
internal audit work would make objective and standard-
ized measurement dif?cult and imperfect, the organization
recognized the potential for de?ciencies in each of the
standardized, objective performance measures. As a result,
the organization authorized supervisors to adjust perfor-
mance one unit up or down to improve objectively-mea-
sured performance.
4
This resulted in an adjusted score. Performance on the
four adjusted measures was then averaged to determine
an overall ‘‘average score,’’ which, in turn, determined
the number of ‘‘shares’’ a manager would receive. These
shares (and a manager’s salary) then determined a man-
ager’s incentive pay amount.
5
For example, one manager re-
ceived values of 4, 3, 3, and 4 on the objective performance
measures. That manager’s supervisor then adjusted the mea-
sures +1, 0, 0, and 0, units, respectively, yielding adjusted
scores of 5, 3, 3, and 4. The manager’s overall average score
was 3.75 [(5 + 3 + 3 + 4)/4], which gave the manager 3
‘‘shares;’’ each share translated into 1.5% of the manager’s
base salary, which, at $74,110, awarded the manager
4
Organizational documents discuss how adjustments can bene?cially
‘‘align individual work with (the organization’s) mission and priorities’’ and
‘‘account for factors that are necessary for effective, ef?cient work
accomplishment’’ that the measures alone may not achieve. The organiza-
tion provided training to supervisors to reinforce this use of subjective
adjustments, and to preclude other uses, including making holistic
judgments (not isolating subjective adjustments on individual measures),
basing them on past performances, and trying to affect bonus payouts.
5
While the pay plan’s ‘‘template’’ permitted unequal weighting of the
four performance measures and a subjective determination of the number
of shares that could be allocated to managers based on ‘‘average scores,’’
the organization’s leaders decided to equally weight the measures and
standardize the speci?c average score ranges that equated to shares (and
then to incentive pay). Therefore, authority to subjectively weight mea-
sures and subjectively decide the number of shares to allocate to managers
was not given to supervisors.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 405
$3335 in incentive pay.
6
Details of the new pay-for-perfor-
mance plan were transparent to all employees. Managers,
for example, knew that supervisors could adjust each of
their performance measures up or down one unit.
This setting is useful for examining whether supervisors
use subjective adjustments to improve objective perfor-
mance measurement using their prior subjective evalua-
tions. The organization’s pay-for-performance plan
explicitly recognized that the objective performance mea-
sures may contain de?ciencies. The proprietary data,
which was not available in prior studies, enables me to di-
rectly measure subjective adjustments to those measures.
Moreover, the incentive system restricted supervisors to
using only subjective adjustments, rather than other forms
of subjectivity, in the evaluation and compensation pro-
cess. Finally, employees received training that reinforced
the use of adjustments to individual performance mea-
sures to improve the system of measures (e.g., the training
identi?ed ‘‘holistic’’ evaluations as improper).
Theory
The inherent complexity of managerial work often pre-
sents dif?culties in specifying and objectively measuring
the relevant dimensions, and desirable outcomes, of man-
agerial effort (Holmstrom & Milgrom, 1991; Hopwood,
1972; Meyer, 2002). Accordingly, objective measures of
managerial performance often fail to indicate a manager’s
contribution toward organizational objectives, distort their
efforts, and provide them with ineffective and/or costly
incentives (Baker, 2000; Baker, Gibbons, & Murphy, 1994;
Milgrom & Roberts, 1992; Prendergast, 1999). Although
the extensive use of objective measures suggests their ben-
e?ts outweigh their costs, these measures are nevertheless
de?cient (i.e., imperfect).
Supervisors, however, observe valuable information
about managerial performance not captured by the avail-
able objective performance measures (Baiman & Rajan,
1995; Fisher, Maines, Peffer, & Sprinkle, 2005). Thus, to
correct for imperfectly measured performance, many orga-
nizations introduce one or more forms of subjectivity into
incentive contracts (MacLeod & Parent, 1999). Subjective
adjustments to objective performance measures are used
to obtain a more accurate representation of a manager’s
contribution toward organizational objectives on particu-
lar dimensions of performance, mitigate distortions in ef-
fort, and improve incentive contracting (Merchant, 1989).
Although subjective adjustments can be bene?cial, they
also entail costs because supervisors’ incentives are often
imperfectly aligned with organizational objectives (Pren-
dergast, 2002; Prendergast & Topel, 1993; Prendergast &
Topel, 1996). Accordingly, it is unclear whether supervisors
will use their discretion over subjective adjustments to
correct for objective measure de?ciencies.
Performance measure properties
According to prior research, objective performance
measures are de?cient when they are (a) not sensitive to
managers’ actions, (b) not congruent with organizational
objectives, (c) noisy, (d) incomplete, and (e) manipulable
(Baker et al., 1994; Banker & Datar, 1989; Datar, Kulp, &
Lambert, 2001; Feltham & Xie, 1994; Holmstrom, 1979;
Holmstrom & Milgrom, 1991). Because performance mea-
sures are imperfect, they all exhibit these de?ciencies to
some degree.
Performance measure sensitivity
A performance measure is sensitive when an agent’s ac-
tion has a large expected effect on it (Banker & Datar,
1989). Measures that are high in sensitivity re?ect agent
effort; thus, they are informative about agent performance
and need no subjective adjustment. Measures that are low
in sensitivity, however, do not manifest agent effort. In
other words, insensitive measures will not always indicate
the level of effort actually put forth by agents. If a certain
(expected) effort level is to be awarded and the measure
is not particularly sensitive, one solution is to adjust the
performance target. If supervisors do not have discretion
over the target, however, another solution is to adjust the
measure, particularly when supervisors perceive the mea-
sure does not appropriately re?ect agent effort.
Performance measure congruity
A measure is congruent when an agent’s action increases
both the measure and the principal’s gross payoff (Feltham
& Xie, 1994). The extent to which a measure is incongruent
determines the extent to which that measure is de?cient.
Thus, an incongruent measure may not always indicate
when the principal’s objectives have been furthered. An
incongruent measure may increase the principal’s gross
payoff but not the score on the performance measure, or
vice versa. One solution is to add more congruent mea-
sures to an agent’s contract, so that the whole contract is
more congruent. Another solution is to adjust the perfor-
mance measure to account for manager contributions to-
ward organizational objectives that the incongruent
measure failed to recognize.
Performance measure precision
A measure exhibits precision (i.e., less noise) when var-
iation in the measure is only minimally attributable to
uncontrollable causes. The extent to which a measure does
not exhibit precision determines the extent to which it is
de?cient. Measures that are low in precision (i.e., high in
noise) are less informative about, and more likely to inac-
curately represent, managerial performance. Supervisors
can, however, correct noisy measures that have been un-
duly in?uenced by uncontrollable factors and circum-
stances by subjectively adjusting them, thereby making
them more accurately re?ect managerial performance
and improving incentive contracting. In contrast, measures
that are high in precision are already informative about
managerial performance and need no subjective
adjustment.
6
Incentive payouts ranged from 0 to $5195 and averaged $1,986 (i.e.,
two shares or 3% of salary). It is worth noting that adjustments and the
resulting overall performance evaluations were meaningful to managers
beyond the provision of one year’s incentive pay. Speci?cally, the evalu-
ations considerably in?uence managers’ promotion potential, opportunities
for training, project assignments, etc.
406 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Performance measure completeness
Completeness refers to the extent to which a measure
captures all relevant dimensions of managerial effort.
Incomplete measures can cause distortions in effort. For
example, managers may exert effort only on those perfor-
mance dimensions measured in their incentive contract
and ignore other important but unmeasured dimensions
(Holmstrom & Milgrom, 1991). A single objective perfor-
mance measure is unlikely to account for all relevant
dimensions. Therefore, multiple performance measures
are often used to evaluate managerial work (Kaplan & Nor-
ton, 1996; Kaplan & Norton, 2001; Lipe & Salterio, 2000).
Ideally, additional measures are incrementally informative
about managers’ efforts (Antle & Demski, 1988; Holm-
strom, 1979), and by expanding the set of implementable
actions, address more performance dimensions. However,
because managerial work is complex, even the use of mul-
tiple performance measures generally fails to capture all
desired and relevant managerial actions, and consequently,
incentive contracts remain incomplete.
Subjective bonuses provide one approach to dealing
with incentive contract incompleteness because they re-
ward managers for performing well on relevant but
unmeasured performance dimensions. However, just as
the entire incentive system can be incomplete, so can indi-
vidual measures in the incentive system, particularly when
they capture entirely different performance dimensions,
such as the number of inventory turns used in the same
contract with the number of days sales outstanding (Mer-
chant, 1989). Thus, by accounting for overlooked perfor-
mance-related actions at the level of individual measures,
subjective adjustments can deal with incompleteness in
each performance measure that fails to address all of the
important aspects of manager performance on that perfor-
mance dimension.
Performance measure manipulability
Some objective performance measures are more sus-
ceptible to manipulation than others. Manipulation occurs
when managers make measures indicate good perfor-
mance without actually producing real performance (Ba-
ker, 1990; Baker et al., 1994).
Measurement properties are theoretically distinct, yet
can be interrelated. For example, the sensitivity of a mea-
sure is only relevant if the measure correlates with the
organization’s payoff, as de?ned by congruity above. Like-
wise, a measure can only be congruent if it is complete
(Feltham & Xie, 1994) because an incomplete measure will
not, by construction, provide incentives for a congruent ef-
fort allocation (Holmstrom & Milgrom, 1991). My de?ni-
tion of completeness focuses on effort allocation, which
presumes a strong de?nition of congruity.
Prior-period performance
Research on ‘‘assimilation effects’’ has shown that cur-
rent evaluations are often in?uenced by prior evaluations
(Smither, Reilly, & Buda, 1988). For example, a new perfor-
mance measurement system that incorporated non-?nan-
cial measures of performance was introduced in Ittner
et al. (2003). Supervisors, however, used their discretion
to prioritize ?nancial performance as had been done in
the past. In Bol and Smith (2011), supervisors made cur-
rent subjective evaluations more consistent with the per-
formance level of a previously-observed objective
measure. Although it is known that prior performance of-
ten in?uences current performance, it is often unclear
why (Kravitz & Balzer, 1992; Murphy, Balzer, Lockhart, &
Eisenman, 1985; Thorsteinson, Breier, Atwell, Hamilton,
& Privette, 2008).
7
I argue that supervisors may make current performance
consistent with prior performance to correct measure de?-
ciencies because prior performance provides supervisors
with an expectation about what current performance
should be (Buda, 1984). Fenner, Lerch, and Kulik (1993)
?nds supervisors are more (less) certain that current per-
formance is appropriately represented when observed cur-
rent performance is (is not) consistent with expectations.
In Reilly, Smither, Warech, and Reilly (1998), evaluating
performance on unfamiliar jobs resulted in greater reliance
on prior performance information. In this paper’s research
setting, prior performance is especially likely to provide
supervisors with an expectation for evaluating current per-
formance because four newobjective measures were intro-
duced as part of the new performance measurement
system implementation. Thus, supervisors had no experi-
ence evaluating these new measures. This lack of familiar-
ity suggests that supervisors may use their prior-period
subjective performance evaluations as a basis to assess
whether current performance (as indicated on the mea-
sures) is appropriately represented.
Importantly, however, whether prior performance pro-
vides a relevant basis for current performance expectations
depends on the relative stability of that performance. Spe-
ci?cally, when performance is stable, deviations in perfor-
mance from one period to another are generally ‘‘small’’—
this means that ‘‘large’’ deviations may be useful indicators
of misrepresented performances. Thus, in circumstances
with stable performance, prior performance information
is particularly relevant as an expectation of current perfor-
mance (Heslin, Latham, & VandeWalle, 2005).
8
In contrast,
when performance is not stable, deviations from one period
to another are large and do not imply performance misrep-
resentations. In such circumstances, prior performance
information does not provide a relevant expectation for cur-
rent performance.
Hypotheses
Several assumptions underlie the ensuing hypotheses. I
assume that supervisors make adjustments in accordance
with their mandate of correcting de?ciencies. This implies
(1) symmetrical (both up and down) adjustments (2) to
individual measures (3) for the purpose of improving
objective performance measurement (4) based on measure
de?ciencies. I also base the hypotheses not on the actual
7
Some of the research assumes and/or investigate whether various
cognitive biases explain assimilation effects. Many of these experimental
studies create arti?cial settings in which the use of prior performance is
intentionally made irrelevant to the current evaluation.
8
This assumption is tested in the empirics section.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 407
de?ciencies, but on supervisor perceptions of the de?cien-
cies. If actual de?ciencies could be observed, a formulaic
system of adjustments could be used in place of the super-
visor’s assessment, which would no longer be required. In-
deed, supervisors do not observe the true, or actual, levels
of sensitivity, congruity, precision, and so on. Moreover,
supervisors’ perceptions are likely to be quite different
than the actual levels—however, the essence of subjectivity
necessitates that adjustments be based on the subjective
perceptions of supervisors.
I assume supervisors maintain ‘‘general’’ perceptions
about the properties of the performance measures because
the measures are standardized across all managers. Stan-
dardization means that performance requirements embed-
ded in each measure are identical across managers.
Nevertheless, because even a poor (good) measure has
the potential of properly (improperly) representing perfor-
mance, I also assume supervisors judge observed manager
performance for each measure using prior-year perfor-
mance. Speci?cally, the more that a manager’s observed
objective performance differs from what supervisors ex-
pect it should be (i.e., ‘‘unexpected performance’’), and
the more that supervisors perceive the objective measure
is incapable (i.e., de?cient or insensitive, incongruent,
imprecise, or incomplete) of properly indicating ‘‘true’’ per-
formance, the more likely supervisors are to conclude that
the objective performance is misrepresented, and in turn,
are more likely to adjust the measure so that it more accu-
rately represents the manager’s ‘‘true’’ performance. In
contrast, an ‘‘ideal’’ (sensitive, congruent, etc.) objective
measure indicating performance that is ‘‘as expected’’
needs no improvement. Thus, the greater the supervisor’s
perception of de?ciency in the objective measures, the
greater the likelihood of adjustments up (down) for unex-
pectedly low (high) objectively-measured manager perfor-
mances. This discussion leads to the following hypotheses:
Hypothesis 1a. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less sensitive supervisors perceive is the objective
measure of performance.
Hypothesis 1b. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less congruent supervisors perceive is the objective
measure of performance.
Hypothesis 1c. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less precise supervisors perceive is the objective
measure of performance.
Hypothesis 1d. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less complete supervisors perceive is the objective
measure of performance.
How will supervisors respond to measures they per-
ceive as manipulable? Manipulability is different than the
other properties. The simplest explanation is that manag-
ers are likely to manipulate performance upward only,
and supervisors are likely to only subjectively adjust per-
formance down to counteract the effects of manipulation.
Thus, the greater the supervisor’s perception that the
objective measures are manipulable, the greater the likeli-
hood of downward adjustments for unexpectedly high
objectively-measured manager performances. This logic
yields the following hypothesis:
Hypothesis 1e. Unexpectedly high objectively-measured
manager performance is more likely to be subjectively
adjusted down by supervisors the more manipulable super-
visors perceive is the objective measure of performance.
Using subjective adjustments for reasons ‘‘other’’ than to
improve objective performance measurement
Despite the hypotheses presented above, management
literature on performance evaluations documents that
supervisors often use evaluation discretion to pursue their
own goals (which do not always match the goals the
organization would like them to pursue) (e.g., Bjerke,
Cleveland, Morrison, & Wilson., 1987; Gibbs et al., 2004;
Longenecker, Sims, & Gioia, 1987). Thus, supervisors may
make current performance consistent with prior perfor-
mance for reasons other than to correct for measure
de?ciencies. One reason may be to in?uence manager
behavior in future periods, such as motivating future
performance, in?uencing retention, and gaining manager
support and cooperation (Bretz, Milkovich, & Read, 1992;
Ilgen, Mitchell, & Frederickson, 1981). For example:
‘‘. . . accuracy was not their primary concern. Rather,
they were much more interested in whether their rating
would be effective in maintaining or increasing the sub-
ordinate’s future level of performance’’ (Longenecker
et al., 1987, p. 187).
Supervisors may make current performance consistent
with prior performance in response to upward in?uence
activities or ingratiatory behavior by managers (Milgrom
& Roberts, 1988; Wayne, Liden, Graf, & Ferris, 1977). Spe-
ci?cally, because supervisors’ use (and non-use) of subjec-
tive adjustments has important effects on managers,
managers are incentivized to in?uence their supervisor’s
subjective assessment of their performance (Ferris & Judge,
1991). This can be costly to organizations because engag-
ing in activities designed to favorably in?uence supervi-
sory assessments diverts managers’ attention and effort
away from more organizationally productive activities
(Anderson, Dekker, & Sedatole, 2010).
Another reason supervisors may make current perfor-
manceconsistent withprior performance involves in?uence
activities of supervisors themselves. Speci?cally, supervi-
sors, in their evaluations of employees, may consider their
supervisors’ evaluation of their supervisory performance.
Because part of supervisors’ jobs includes developing their
subordinates, supervisors may use adjustments to project
the image that they are skilled leaders, increasing their po-
tential for promotion to higher levels of responsibility,
power, and control in the organization (Ilgen & Favero,
1985; Ilgen et al., 1981; Welch & Welch, 2006).
408 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Finally, supervisors may make current performance
consistent with prior performance due to favoritism,
whereby supervisors reward some managers over others
based on their personal preferences rather than on these
managers’ performance (Prendergast & Topel, 1996).
Supervisors apply favoritism in part because their incen-
tives with respect to their evaluation of managers are not
always perfectly aligned with organizational objectives.
Speci?cally, supervisors occupy just one of the multi-lay-
ered agency relationships existing in most hierarchical
organizations, and thus are not themselves full residual
claimants on manager output (Bol, 2008). As such, supervi-
sors may bene?t from applying favoritism while the orga-
nization bears the cost. Thus, supervisors may adjust
measures of a manager who is favored, even when actual
performance does not justify the adjustment.
In sum, if supervisors use adjustments to make current
performance consistent with prior performance for reasons
other than to correct de?ciencies in the objective mea-
sures, unexpectedly high (low) objectively measured per-
formance in the current period is less (more) likely to be
subjectively adjusted downward (upward). I therefore po-
sit the following hypotheses:
Hypothesis 2a. Unexpectedly low objectively measured per-
formance is more likely to be subjectively adjusted upward.
Hypothesis 2b. Unexpectedly high objectively measured per-
formance is less likely to be subjectively adjusted downward.
Summary of hypotheses
To summarize, if supervisors use adjustments to make
current performance consistent with prior performance
to improve the measurement of individual objective mea-
sures, they should be more likely to subjectively adjust
objective performance measures under certain conditions.
Supervisors should adjust objective measures upward
(downward) when managers’ performance on each of
those measures is unexpectedly low (high) and supervisors
perceive the measures are de?cient with respect to sensi-
tivity (H1a), congruity (H1b), precision (H1c), and com-
pleteness (H1d). Supervisors should also be more likely
to adjust unexpectedly-high objective measures down-
ward the more manipulable (H1e) they perceive the mea-
sure to be. If supervisors use adjustments to make current
performance consistent with prior performance for reasons
other than to correct de?ciencies in the objective mea-
sures, unexpectedly low (high) performances should be
more likely to be subjectively adjusted up (down) (Hypoth-
eses 2a and 2b, respectively).
Variable measurement and empirical speci?cation
Data collection and survey development
I collected data to test the hypotheses from three
sources. First, I collected the proprietary evaluation and
compensation data for all managers for the 2006 evalua-
tion year from the organization with oversight of the
pay-for-performance plan. Second, I collected the prior
2 years (2004 and 2005) of performance evaluation data,
as well as turnover data for the 2007 evaluation year, from
the internal audit organization. Third, I developed surveys
directed at three layer in the organization hierarchy: (1)
managers, (2) supervisors, and (3) division managers
(supervisors of the supervisors).
9
An organizational repre-
sentative helped coordinate and review the surveys, which
measured supervisor perceptions of the extent to which
the objective performance measures were de?cient and
how supervisors used adjustments.
The surveys were conducted using Internet survey
software.
10
I obtained complete survey responses from 8
of the 12 division managers (67%), 19 of the 36 supervisors
(53%), and 39 of the 130 managers (30%). All 12 divisions
were represented in the 19 supervisor responses; the mean
number of supervisor responses per division was 1.6 and
ranged from 1 to 3 (SD = 0.9). Similarly, all divisions were
adequately represented in the distribution of the entire
set of responses. Speci?cally, the mean number of re-
sponses per division was 5.5 and ranged from 4 to 9
(SD = 1.4).
Measures
An indicator variable is set to 1 (and 0 otherwise) to
measure the presence of a subjective adjustment by a gi-
ven supervisor to a particular performance measure of a gi-
ven manager.
11
Unexpected manager performance
Prior-year (2005) subjectively-measured performance
provides a baseline measure of supervisors’ expectations
of managers’ current-year (2006), pre-adjusted, objec-
tively-measured performance. Because performance across
years was evaluated on different scales, I calculate z-scores
and use the difference in standardized performance scores
across years to measure unexpected performance
(Unexp_Perf):
P
ij
ÀP
^
i
d
i
À
O
j
ÀO
^
d
ð1Þ
where P
ij
is performance on measure i for manager j, and P
^
i
and d
i
are the mean and standard deviation, respectively, of
all pre-adjusted performances on measure i in 2006. O
j
is
the overall evaluation for manager j, and O
^
and d are the
mean and standard deviation, respectively, of all overall
manager performances in 2005. Higher (lower) values in
9
I used the supervisor survey for my independent variables; the audit
and division manager surveys were used mostly for validation purposes.
10
Emails were sent to administrative assistants, who forwarded separate
emails describing the purpose of the survey to division managers,
supervisors, and managers. The emails also contained the survey links,
which, the respondents were informed, would be accessible from March 5
to March 23, 2007.
11
Nineteen supervisors completed the survey in its entirety, thus
providing me with their perceptions of the extent to which the objective
performance measures were de?cient. I have prior-year performance for 68
managers who worked for these 19 supervisors. Thus, the hypotheses use
the 272 performance measures of the 68 managers who worked for the 19
supervisors who completed the survey.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 409
Eq. (1) indicate current-year performance is higher (lower)
than prior-year performance. I divide the sample into three
groups. If Eq. (1) > 1, unexpectedly high performance (Un-
exp_High_Perf) takes the value of 1, and 0 otherwise. If
Eq. (1) < À1, unexpectedly low performance (Unexp_Low_-
Perf) takes the value of 1, and 0 otherwise. If Eq. (1) is be-
tween À1 and 1, expected performance (Exp_Perf) takes the
value of 1, and 0 otherwise. This procedure resulted in clas-
sifying 48 (18%) pre-adjusted measures as unexpectedly
high, 49 (18%) as unexpectedly low, and 175 (64%) as
expected.
Several assumptions underlie how well deviations
from the prior-year represent current-year unexpected
performance. I examine the reasonableness of these
assumptions. First, to the extent that manager perfor-
mances vary year-to-year, supervisors may not have sta-
ble expectations of manager performance (Heslin et al.,
2005). Correlating 2005 subjectively-evaluated perfor-
mance with 2004 subjectively-evaluated performance
reveals a correlation of 0.77 (p < .01, two-tailed). This
high correlation indicates that manager performance,
as subjectively assessed by supervisors during the
2 years prior to the objective system implementation,
is remarkably consistent year-on-year. As such, it pro-
vides strong support to the idea that performance is
very stable, or at least that supervisors believe it is very
stable. Accordingly, I conclude that prior-year (2005)
subjectively-measured performance appears to be a rea-
sonable baseline of supervisors’ expectation of manag-
ers’ 2006 objectively-measured performance, and may
be particularly useful for supervisors in assessing cur-
rent period performance as indicated on the objective
measures.
Second, because different supervisors may differentially
subjectively assess managers, another assumption is that
supervisor-manager pairs remain relatively constant from
2005 to 2006. Of the 39 managers surveyed (some of
whom are in my sample tests and some of whom are
not), 37 (95%) indicated they were evaluated by the same
supervisor in 2005. Thus, supervisor-manager pairs appear
to be stable in this organization. I have no reason to believe
a different level of stability in supervisor-manager pairs
exists in my sample. To the extent supervisor–manager
pairs ?uctuated across years more frequently than my sur-
vey respondents indicate, I believe it only creates noise, not
bias, in my tests.
Finally, I assume the overall subjective evaluation
supervisors gave to managers in 2005 proxies for their
expectation of managers’ 2006 performance on each of
the four objective measures. An examination of appraisal
records indicates supervisors did not distinguish across
performance dimensions when they subjectively evaluated
managers in the prior year.
12
In other words, supervisors
did not demonstrate re?ned subjective beliefs about manag-
ers on separate dimensions in prior years, but rather held a
global prior subjective impression. Thus, it is unlikely super-
visors would have subjective expectations in the current
year that are unique to each of four new objective measures
(for which they have no experience observing). Given such
evidence in this organization, as well as widespread evi-
dence in other contexts of supervisors’ tendencies to rate
managers similarly across different subjective performance
dimensions (e.g., Murphy & Cleveland, 1991, 1995), this
assumption seems reasonable but is nevertheless made
explicit.
Performance measure property perceptions
I use survey questions adapted from Moers (2006),
Gibbs et al. (2004), and Anderson et al. (2010) to measure
supervisors’ perceptions
13
of the extent of de?ciency of
each performance measure. I use a seven-point, fully-an-
chored scale to indicate the extent to which each perfor-
mance measure re?ects the following properties:
sensitivity (H1a), congruity (H1b), noise (H1c), complete-
ness (H1d), veri?ability (H1e), and manipulability (H1f).
14
Thus, each supervisor answered questions related to perfor-
mance measure properties for each of the four performance
measures.
I construct the performance measure property vari-
ables by performing common factor analysis with vari-
max rotation
15
to identify the underlying properties of
performance measures as perceived by the 19 supervisors.
I pool the data across the measures to yield a ?nal factor
analysis sample of 76 cases (19 supervisor responses  4
performance measures). The resulting factor analysis used
9 questions
16
and identi?ed 4 factors
17
: (1) sensitive/con-
gruent,
18
(2) complete, (3) noise, and (4) manipulable.
These 4 factors explained 85% of the total variance, had
factor loadings ranging from |0.76| to |0.95|, and Cron-
bach alphas from 0.84 to 0.93. See Table 1, Panel A, for
12
For example, one manager with an overall evaluation of 70 (76)
received scores of either 7 or 8 only (8 or 9 only) across the 9 performance
dimensions.
13
I use supervisor perceptions in my statistical tests because my theory is
that supervisors—not managers—make adjustments based, in part, on their
perceptions (not the actual degree) of performance measure property
de?ciencies. However, I also collected division and audit manager percep-
tions to analyze differences in perceptions across the organization hierar-
chy. While there were no differences between division manager and
supervisor perceptions, audit manager perceptions of the performance
measure properties diverged considerably from those of supervisors. In
particular, managers perceived three of the four performance measures as
being signi?cantly more de?cient than supervisors did concerning all but
one performance measure property, that property being different for each
measure.
14
Two questions (which organizational representatives requested I
include) addressed whether the performance measures captured the
quality and quantity of managerial performance. Because various factor
analyses revealed that they did not correlate highly with the other
questions to which they were a factor, I do not discuss these questions
further. I also do not further discuss two survey questions that addressed
measure veri?ability, which I included in the survey before I knew the
measures were indeed veri?able.
15
I used varimax rotation for my ?nal factor analysis to achieve
discriminant validity across the theoretically distinct constructs.
16
I dropped one question related to congruity because it had low cross-
loadings on two factors.
17
The most interpretable factor structure resulted when I retained factors
with eigenvalues >0.9 even though the conventional, though arbitrary, cut-
off value is usually 1.0.
18
Sensitivity and congruity may have loaded on the same factor because:
(1) supervisors cannot distinguish between sensitivity and congruity, (2)
sensitivity and congruity are highly correlated, and/or (3) my survey
questions failed to distinguish between sensitivity and congruity.
410 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
descriptive statistics of the questions and Panel B for the
factor analysis.
Controls
Because older supervisors generally have more expe-
rience and less career concerns than younger supervi-
sors, older supervisors may trust their subjective
assessments more and be less inhibited with their use
of adjustments than younger supervisors (Gibbons &
Murphy, 1992).
19
Using archival data, I use supervisor
age (Sup_Age) to control for differential use of adjust-
ments. In addition, because similar genders could affect
the use of subjective adjustments (Bol, 2011), I include
a binary variable that captures whether there is a gender
difference between managers and supervisors (Dif_Gen).
Table 2 reports descriptive statistics of unexpected per-
formance and control variables, and Table 3 reports a cor-
relation matrix of all independent variables in the main
analysis.
Empirical speci?cation
To test the hypotheses, I estimate two probit models in
Eq. (2) below:
Table 1
Descriptive statistics and factor analysis of the performance measure property survey items.
Panel A: Descriptive statistics of the performance measure property survey items aggregated across the 4 performance measures (n = 76)
Mean
Std.
dev. Range
Sensitive/congruent
a. If AMs perform well, it is directly re?ected in better performance on this performance measure 5.37 1.42 1–7
b. AM effort leads to better performance on this performance measure 6.07 1.02 1–7
c. The (organization’s) missions and goals are further accomplished when AMs perform well on this performance
measure
5.89 1.30 1–7
d. When AMs perform well on this performance measure, they contribute to the (organization’s) mission and goals 5.88 1.22 1–7
Cronbach’s alpha = .87
Complete
e. This performance measure does not capture important Performance measure name activities that AMs perform
a
4.34 2.04 1–7
f. This performance measure captures all the dimensions of effort that are required for AMs in this division to perform
well on this performance measure
4.01 1.97 1–7
Cronbach’s alpha = .84
Noise
g. AM performance on this performance measure is affected by unanticipated events/changes 4.79 1.90 1–7
h. AM performance on this performance measure is in?uenced by things outside of their control 4.68 1.84 1–7
Cronbach’s alpha = .93
Manipulable
i. AMs can manipulate this performance measure to ensure they meet performance goals without actually performing
well
3.47 1.69 1–7
Panel B: Common factor analysis with varimax rotation of the performance measure property survey items (all factor loadings
greater than .3 are shown, n = 76)
Items
Sensitive/
Congruent Complete Noise Manipulable
a. The (organization’s) missions and goals are further accomplished when AMs perform well on
this performance measure
0.88
b. If AMs perform well, it is directly re?ected in better performance on this performance measure 0.77
c. When AMs perform well on this performance measure, they contribute to the (organization’s)
mission and goals
0.80
d. AM effort leads to better performance on this performance measure 0.76
e. This performance measure does not capture important Performance measure name activities
that AMs perform
a
0.88
f. This performance measure captures all the dimensions of effort that are required for AMs in
this division to perform well on this performance measure
0.90
g. AM performance on this performance measure is affected by unanticipated events/changes 0.95
h. AM performance on this performance measure is in?uenced by things outside of their control 0.95
i. AMs can manipulate this performance measure to ensure they meet performance goals without
actually performing well
À0.88
Eigenvalues 3.74 2.13 1.18 0.92
AMs = audit managers.
a
This item was reverse-coded.
19
The archival measure of supervisor age is signi?cantly correlated with
the self-reported survey measure of experience (0.40, p < 0.05, one-tailed,
n = 19).
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 411
A
ijk
¼ b
0
þb
1
Unexp Perf
ij
þb
2
Insensitive Incongruent
ik
þb
3
Noise
ik
þb
4
Incomplete
ik
þb
5
Manipulable
ik
þb
6
Sup Age
k
þb
7
Dif Gen
jk
þb
8
Insensitive Incongruent à Unexp Perf
ijk
þb
9
Noise à Unexp Perf
ijk
þb
10
Incomplete
à Unexp Perf
ijk
þb
11
Manipulable à Unexp Perf
ijk
ð2Þ
in which the dependent variable in the ?rst (second) probit
model is an upward (downward) subjective adjustment (A)
to performance measure i for manager j of supervisor k,
that takes on a value of 1 if the performance measure is ad-
justed up (down) and 0 otherwise. Unexp_Perf in the up-
ward (downward) model is measured as unexpectedly
low (high) performance (i.e., Unexp_Low_Perf and Unex-
p_High_Perf). I code all performance measure properties
in negative terms, so that higher (lower) values indicate
more (less) de?ciency. That way, the coef?cients on the
interaction terms should always be positive. If supervisors
make adjustments consistent with improving objective
measurement, then upward (downward) adjustments
should be more positively related to unexpectedly low
(high) performances the more de?cient are the measures.
In both models, I use Huber–White standard errors cor-
rected for the supervisor cluster.
20
Results
Descriptive statistics
Before examining the results, Table 4 provides descrip-
tive statistics. Of the 272 measure-speci?c observations, 85
(31%) were adjusted; 81 (95%) of these adjustments were
upward. As can be seen in Table 4, Panel B, Program Man-
agement was the most often adjusted measure. The score
of ‘‘3’’ was the most frequent pre-adjusted score across
all measures, and adjustments were made to only pre-ad-
justed scores of 3 and 4. Table 4, Panel C, shows that the
use of adjustments varied greatly across supervisors.
Supervisors, on average, gave 4.5 adjustments, ranging
from 1 and 11 (SD = 2.8).
21
Table 5, Panel A, shows that adjustments varyby manager.
Of the 68 sample managers, 47 (69%) received at least one
adjustment. Adjustmentsincreased, byalmost 60%, theincen-
tive pay of 40 (85%) of the 47 managers to whomsupervisors
gave at least 1 adjustment. In addition to the meaningful im-
pact on managers’ promotability, opportunities for training,
project assignments, and so on, Panel B shows that adjust-
ments also materially impact incentive pay.
Table 6 reports the correlations between prior-year
(2005) overall subjective evaluations and current-year
(2006) pre- and post-adjustedperformances. The magnitude
of the correlation with the pre-adjusted objective measures
(0.19–0.34) is similar to the mean correlations between
objective and subjective measures reported in other studies
(Bommer, Johnson, Rich, Podsakoff, &MacKenzie, 1995; Hen-
eman, 1986). Prior-year subjective evaluations were posi-
tively correlated with current-year pre-adjusted overall
performance and the Planning, Program Management, and
Professional Development (but not the Reporting) measures;
prior-year subjective evaluations were positively correlated
with all post-adjusted measures as well as the overall post-
adjusted performance. Comparing the pre- and post-ad-
justed correlations with each other shows that the correla-
tions with post-adjusted measures are higher for the
overall evaluation and all individual measures except mea-
sure one (Planning). Thus, supervisors’ use of subjective
adjustments increased the correlation with prior-year sub-
jectively-measured performance for managers’ overall per-
formance and for the Program Management, Reporting, and
Professional Development measures (andactually decreased
it for the Planning measure).
Main analysis
The results in Panel A (B) of Table 7 are used to test the
hypotheses for upward (downward) adjustments.
22,23
Table 2
Descriptive statistics of unexpected performance and control variables.
Variable n Mean Std. dev. Range
Unexp_Low_Perf 272 0.18 0.39 0–1
Unexp_High_Perf 272 0.18 0.38 0–1
Exp_Perf 272 0.64 0.48 0–1
Unexp_Low_OV_Perf 68 0.15 0.36 0–1
Dif_Gen 68 0.51 0.50 0–1
Sup_Age 19 47.8 6.34 38–58
Unexpected performance is measured as:
P
ij
ÀP
^
i
d
i
n o
À
O
j
ÀO
^
d
n o
ð1Þ
where P
ji
is performance on measure i for manager j, and P
^
i
and d
i
is the
mean and standard deviation, respectively, of all pre-adjusted perfor-
mances on measure i in 2006. O
j
is the overall evaluation for manager j,
and O
^
and d the mean and standard deviation, respectively, of all overall
manager performances in 2005. If Eq. (1) > 1, unexpectedly high perfor-
mance (Unexp_High_Perf) takes the value of 1 and 0 otherwise. If Eq.
(1) < À1, unexpectedly low performance (Unexp_Low_Perf) takes the value
of 1 and 0 otherwise. If Eq. (1) is between À1 and 1, expected performance
(Exp_Perf) takes the value of 1, and 0 otherwise.
Unexpectedly low overall manager performance (Un_Low_OV_Perf) is
measured as:
P
j
ÀP
^
d
n o
À
O
j
ÀO
^
d
n o
ð4Þ
where P
j
is manager j’s performance, and P
^
and d is the mean and
standard deviation, respectively, of all pre-adjusted overall performances
in 2006. O
j
is the overall evaluation for manager j, and O
^
and d is the
mean and standard deviation, respectively, of all overall manager per-
formances in 2005. If Eq. (4) < À1, unexpectedly low overall performance
(Unexp_Low_OV_Perf) takes the value of 1 and 0 otherwise.
Dif_Gen is a binary variable that takes the value of 1 if there is a gender
difference between managers and supervisors, and 0 otherwise. Sup_Age
is the supervisor’s age as provided per organizational records.
20
When there are nested levels of clustering, Cameron and Miller (2011,
p. 14) advise to cluster on the more aggregate group to appropriately
correct standard errors.
21
This mean was not signi?cantly different from the mean of 3.12
(standard deviation of 2.96) for supervisor non-respondents (p > .05, two-
tailed), leading me to believe that survey respondents do not differ in
important ways from non-respondents.
22
Variance In?ation Factors (VIFs) for all models presented in this paper
indicate a lack of multicollinearity.
23
Because interaction terms in nonlinear models can be dif?cult to
interpret (Ai & Norton, 2003), I introduce interaction terms individually and
calculate z-statistics on interaction terms following Norton, Wang, and Ai
(2004).
412 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Upward adjustments
The results in Panel A of Table 7 support H1c, H1d, and
H2 (but not H1a/b) for upward adjustments. The coef?-
cient on the Unexp_Low_Perf  Insens_Incong interaction
term in Column 2 is positive but not signi?cant, and thus
H1a/b is not supported. The positive and signi?cant coef?-
cient for the Unexp_Low_Perf  Noise interaction term in
Column 3 supports H1c. Following Greene (2010), Panel
A of Fig. 1 provides graphical evidence that incorporates
the main effects of Unexp_Low_Perf and Noise, and their
interaction from the model in Column 3 of Table 7, Panel
A. The graph shows that increasing performance measure
noise from one standard deviation below to one standard
deviation above the mean increases the estimated pre-
dicted probability of upward adjustments for unexpectedly
low performances by approximately 31%, as compared
with an increase of approximately 2% for performances
that are not unexpectedly low.
24
The difference between
these increases is economically signi?cant and supports
the signi?cance of the Unexp_Low_Perf  Noise interaction
(H1c).
Table 5
Examination of how many adjustments managers generally received and
whether receiving adjustments materially affected manager pay.
Panel A: Adjustments by manager
#
Managers with 1 adjustment 20
Managers with 2 adjustments 19
Managers with 3 adjustments 5
Managers with 4 adjustments
3
Total managers with 1 or more adjustment 47
Managers with 0 adjustments
21
Total managers 68
Mean # of adjustments per manager = 1.25
Panel B: Materiality of adjustments
# of
Managers
Incentive pay
before
adjustment(s)
Incentive pay
after
adjustment(s)
Change in
incentive
pay (%)
Adjustment(s)
did change
incentive
pay
40 $62,000 $98,464 58.8
Adjustment(s)
did NOT
change
incentive
pay
7 $10,150 $10,150
0
Total Managers
with 1 or
more
Adjustments
47 $72,150 $109,515 50.5
Table 4
Descriptive information of subjective adjustment use.
Panel A: Distribution of pre-adjusted performance scores (by each
objective measure)
Pre-adjusted score
2 3 4 5 Total
PM1 1 52 12 3 68
PM2 0 45 17 6 68
PM3 1 43 20 4 68
PM4
1 47 19 1 68
Total 3 187 68 14 272
Panel B: Distribution of adjustments (by pre-adjusted performance
score and objective measure)
2 3
a
4 5 Total
PM1 0 5 4 0 9
PM2 0 26 12 0 38
PM3 0 14 7 0 21
PM4
0 13 4 0 17
Total 0 58 27 0 85
Panel C: Distribution of supervisor use of adjustments
Supervisors that made: #
0 or 1 adjustment 2
2 or 3 adjustments 8
4 or 5 adjustments 1
6 or 7 adjustments 6
8 or 9 adjustments 1
10 or 11 adjustments
1
Total # of supervisors 19
PM1 = Planning measure.
PM2 = Program Management measure.
PM3 = Reporting measure.
PM4 = Professional Development measure.
Mean # of adjustments per supervisor = 4.5.
a
All four downward adjustments (3 to PM2 and 1 to PM3) were made
to a pre-adjusted score of 3. Thus, upward (downward) adjustments were
made only to pre-adjusted scores of 3 and 4 (3).
Table 3
Correlation matrix of the independent variables used to examine whether subjective adjustments were used to improve individual objective measurement.
Variable 1 2 3 4 5 6 7
1. Unexp_Low_Perf
2. Unexp_High_Perf À0.22
3. Sensitive_Congruent À0.04 0.10
4. Noise 0.01 0.00 0.00
5. Complete À0.01 0.02 0.00 0.00
6. Manipulable 0.14 À0.07 0.00 0.00 0.00
7. Sup_age À0.01 À0.14 À0.13 À0.03 À0.07 À0.09
8. Dif_gen 0.13 0.01 À0.01 À0.03 À0.05 0.11 À0.06
Bolded Pearson correlation coef?cients are signi?cant at the 10% level (two-tailed).
24
Measure properties are factor scores and thus have a mean of 0 and
standard deviation of 1.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 413
The positive and signi?cant coef?cient on the
Unexp_Low_Perf  Incmplt interaction in Column 4 sup-
ports H1d. Panel B of Fig. 1 incorporates the main effects
of Unexp_Low_Perf and Incompleteness, and their interac-
tion from the model in Column 4 of Table 7, Panel A. The
graph shows that increasing performance measure incom-
pleteness from one standard deviation below to one stan-
dard deviation above the mean increases the estimated
predicted probability of upward adjustments for unexpect-
edly lowperformances by approximately 27%, as compared
with a decrease of approximately 9% for performances that
are not unexpectedly low. The difference between these is
economically signi?cant and supports the signi?cance of
the Unexp_Low_Perf  Incompleteness interaction (H1d).
The graphs in Panels A and B of Fig. 1 also provide evi-
dence consistent with H2a. The regression line for objec-
tively-measured performances that are unexpectedly low
lies above the line for performances that are not unexpect-
edly low. Thus, on average, unexpectedly low performance
has a positive and signi?cant main effect. To properly
interpret the positive coef?cient on Unexp_Low_Perf, fol-
lowing Greene (2010), I examine the change in probability
of adjustment by moving the indicator variable for unex-
pectedly low performance from 0 to 1 in settings of low,
medium, and high levels of de?ciency (with respect to Inc-
mplt and Noise).
25
At low levels of de?ciency, I do not ?nd a
signi?cant effect. However, at medium levels of Noise
(Incompleteness), the change to unexpectedly low perfor-
mance increases the estimated predicted probability of up-
ward adjustments by approximately 17% (20%). At high
levels of Noise (Incompleteness), the change to unexpectedly
low performance increases the estimated predicted proba-
bility of upward adjustments by approximately 32% (37%).
Thus, in support of H2a, unexpectedly low performance
has a positive and signi?cant main effect when measures
have medium or high levels of de?ciency. Overall, the results
suggest that unexpectedly low current performances were
raised to correct measure de?ciencies, as well as for other
reasons.
26
Downward adjustments
The results in Panel B of Table 7 do not support H1 or
H2 for downward adjustments. Results are generally in
the predicted direction, but are not signi?cant. For exam-
ple, for the model in Column 5 of Panel B of Table 7, the
coef?cient on the Unexp_High_Perf  Manipulability inter-
action (H1e) is positive as predicted, but insigni?cant. Fol-
lowing Greene (2010), increasing performance measure
manipulability from one standard deviation below to one
standard deviation above the mean increases the esti-
mated predicted probability of downward adjustment for
unexpectedly high performances by approximately 9%, as
compared with a decrease of approximately 4% for perfor-
mances that are not unexpectedly high. Similarly modest
effects are observed with the other interactions.
To properly interpret the coef?cients on Unexp_High_-
Perf and test H2b, following Greene (2010), I examine the
change in probability of downward adjustments by moving
the indicator variable for unexpectedly high performance
from 0 to 1 in settings of low, medium, and high levels of
de?ciency.
27
For the model in Column 5 of Panel B, at low
levels of Manipulability, there is no effect on the estimated
predicted probability of downward adjustments when the
indicator variable on unexpectedly high performance is
changed from 0 to 1. At medium (high) levels of Manipulabil-
ity, the change increases the estimated predicted probability
of downward adjustments by approximately 6% (12%). Sim-
ilar modest effects are observed with the other models in Ta-
ble 7, and thus do not support H2b. Because of the small
sample size (n = 272) and rarity of downward adjustments
(4), I conclude that traditional statistical tests lack suf?cient
power to unequivocally detect whether downward adjust-
ments were used to correct de?ciencies.
Supplemental analyses of upward adjustments
Table 8 reports additional tests of whether upward
adjustments were made to correct de?ciencies in the
objective measures.
28
Column 1 of Table 8 incorporates all
interaction terms from Table 7, Panel A, into one model. Col-
umn 2 of Table 8 then adds a new indicator variable for
unexpectedly high performances (Unexp_High_Perf) to test
for the possibility that unexpectedly high performances are
not likely to be adjusted upward, and are thus driving the re-
sults in the main analysis.
29
The results indicate that unex-
Table 6
Prior-year (2005) subjectively-measured performance correlated with
current-year (2006) pre- and post-adjusted objectively-measured
performance.
Variable 2006 Objective performance
Pre-adjusted Post-adjusted
1. PM1 0.34
***
0.22
*
2. PM2 0.31
**
0.41
***
3. PM3 0.19 0.42
***
4. PM4 0.22
*
0.32
***
5. Overall 0.26
***
0.33
***
PM1 = Planning measure.
PM2 = Program Management measure.
PM3 = Reporting measure.
PM4 = Professional Development measure.
*
Bolded Pearson correlation is statistically signi?cant at the 10% level.
**
Bolded Pearson correlation is statistically signi?cant at the 5% level.
***
Bolded Pearson correlation is statistically signi?cant at the 1% level.
25
Measures with a factor score on incomplete >0 (0) or the most incomplete (>0) measures, the coef?cient
on unexpectedly low performance remains signi?cant, and
the only signi?cant interaction (marginally) is with
favoritism.
38
Thus, supervisors make unexpectedly low current per-
formances consistent with prior performance ‘‘for other
reasons’’ only when the measures have medium or high,
but not low, de?ciency levels. Consistent with prior studies
in accounting (e.g., Tayler, 2010) and psychology (Kunda,
1990), the evidence indicates that supervisors use discre-
tion to pursue their own goals only when measures with
some de?ciency (‘‘medium’’ levels) provide them with
the leeway they need to justify giving adjustments for
the ‘‘other reasons’’ examined in this study. Measures with
low levels of de?ciency apparently prevent supervisors
from justifying adjustments for other reasons. At high lev-
els of de?ciency, adjustments to make current perfor-
mance consistent with prior performance may have been
made for still other reasons not examined in this study.
Manager level analysis
Because of the interpretation dif?culties of interaction
terms in nonlinear models, Table 11 reports the results of
an OLS regression in which the dependent variable is the
percent of adjustments made to the measures of a given
manager. Necessarily, the analysis is at the level of the
individual manager, and unexpected performance is calcu-
lated as before except at the manager level (Unexp_Lo-
w_OV_Perf).
39
Although the analysis is similar to that in
Table 10, it is important to note that it examines a subtly dif-
ferent question. Speci?cally, instead of examining whether
supervisors used adjustments to align managers’ individual
measures with their prior performance, it examines how
many adjustments supervisors used to align managers’ cur-
Table 11
OLS regressions of upward adjustments for ‘‘other’’ reasons at the manager
level (n = 68).
(1) (2)
Coeff. t-Stat Coeff. t-Stat
Intercept 0.049 0.20 À0.038 À0.15
Unexp_Low_OV_Perf [1] À0.055 À0.79 À0.129 À2.35
**
Future_Focus [2] 0.065 1.68 0.058 1.51
Mgr_In?uence [3] 0.055 1.96
*
0.054 2.03
*
Sup_In?uence [4] À0.032 À1.00 À0.033 À0.92
Favoritism [5] 0.040 1.42 0.027 1.00
Sup_age 0.006 1.08 0.008 1.35
Dif_gen À0.028 À0.42 À0.036 À0.55
[1] X [2] 0.115 2.09
**
[1] X [3] 0.162 2.79
***
[1] X [4] 0.079 1.24
[1] X [5] 0.135 3.59
***
Adj. R
2
0.18 0.20
Dependent variable = percent of performance measures adjusted upward.
Huber–White standard errors are clustered by supervisor. For variable
de?nitions, see Table 9 (Panel A) and Table 2.
*
Statistically signi?cant different at the 10% level.
**
Statistically signi?cant different at the 5% level.
***
Statistically signi?cant different at the 1% level. (one-tailed for
interaction terms and two-tailed otherwise).
38
As additional con?rmation checks, in untabulated analyses in which
the sample is restricted to the most complete (precise) measures, the main
effect of unexpectedly low performance is unrelated to upward adjust-
ments, and no interactions (except its interaction with favoritism) are
signi?cant.
39
These tests use the 68 managers who worked for the 19 supervisors.
420 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
rent overall performance with their prior performance. Hu-
ber–White standard errors, corrected for the supervisor
cluster, are used.
In Model 2 of Table 11, the coef?cients on interac-
tions between unexpectedly low performance and future
focus, manager in?uence, and favoritism are all signi?-
cant and positive. The results imply that when manag-
ers’ current-year objectively-measured performance is
unexpectedly low, supervisors make more upward
adjustments to in?uence managers’ future behavior, in
response to manager in?uence activities, and to apply
favoritism. Note, however, in Model 1 of Table 11, that
unexpectedly low overall performance was not posi-
tively related to the percentage of upward adjustments,
suggesting supervisors do not raise multiple measures to
make unexpectedly low current overall performances of
managers be consistent with prior overall performances
of managers.
40
Post hoc analyses of downward adjustments
This section reports the results of several post hoc anal-
yses that examine other reasons for the use, and non-use,
of downward adjustments.
Turnover
I analyze whether supervisors used downward adjust-
ments, at least partially, to prompt managers to leave the
organization. Of the 111 managers in the 2006 evaluation
year cycle for whom I have complete evaluation records,
a total of 10 downward adjustments were given to 7
managers.
41
Of these 111 managers, 4 left during the 2007
evaluation year. Of the 4 who left, 3 had received a down-
ward adjustment in 2006 (2 had received 1 downward
adjustment, and 1 had received 2 downward adjustments).
Table 12, Panel A, estimates a probit model of whether
the likelihood of leaving in 2007 is a function of having
performance lowered in 2006. The results suggest that one
Table 12
Post hoc analyses of other reasons for making, and avoiding, downward adjustments.
Panel A: Probit Regression of the effect of making downward adjustments on manager turnover (n = 111).
(1) (2) (3)
Left_Org
t+1
= B
0
+ B
1
One_Down_Adj
t
+ B
2
Several_Down_Adj
t
+ e
t
Intercept À2.070 À1.919 À2.070
(À7.42)
***
(À7.95)
***
(À7.42)
***
One_Down_Adj
t
1.504 1.228
(2.95)
***
(2.17)
**
Several_Down_Adj
t
1.919 0.842
(2.06)
**
(0.82)
Pseudo R
2
0.18 0.12 0.20
Panel B: Reasons for avoiding downward adjustments
Supervisors who
avoided down
adjustments
(n = 16)
Supervisors who
gave down
adjustments
(n = 3)
t-Test for mean
differences
Mean
(std.
dev.)
Range Mean
(std.
dev.)
Range p-Value
I avoided giving negative adjustments because it:
1. Might have resulted in negative consequences for AMs (e.g., demotion, no bonus,
salary freeze, etc.)
2.37 1–7 1.33 1–2 0.06
*
(2.13) (0.58)
2. Might have threatened the self-esteem of AMs 2.38 1–7 1.33 1–2 0.04
**
(1.75) (0.58)
3. Would have resulted in a written record of poor performance 2.00 1–5 1.33 1–2 0.20
(1.32) (0.58)
4. May be viewed as a failure on my part to be a good supervisor 1.81 1–4 1.00 1–1 0.01
***
(1.17) (0.00)
Dependent variable (Left_Org
t+1
) = 1 if manager left the organization during the 2007 evaluation year, and 0 otherwise. One_Down_Adj
t
= 1 if manager had
one measure adjusted downward in the 2006 evaluation year, and 0 otherwise. Several_Down_Adj
t
= 1 if manager had two or more measures adjusted
downward in the 2006 evaluation year, and 0 otherwise. Standard errors are adjusted for the supervisor cluster.
AMs = audit managers.
*
Statistically signi?cant different at the 10% level.
**
Statistically signi?cant different at the 5% level.
***
Statistically signi?cant different at the 1% level.
40
In untabulated analyses, I also tested whether supervisors used
adjustments to correct de?ciencies at the manager level (using a probit
model) and how many (using an OLS model with percent of adjustments as
the dependent variable). Empirical evidence is more supportive of super-
visors correcting de?ciencies at the level of each measure.
41
Five managers received one downward adjustment, one manager
received two downward adjustments, and one manager received three
downward adjustments.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 421
downward adjustment was suf?cient to prompt individuals
to leave the organization. Although I do not know whether
these managers were ?red, retired, found a different job,
etc. (i.e., I only know whether they left the organization dur-
ing 2007), the results are consistent with one of the over-
arching goals of the new plan—to retain the best
performers.
42
Avoiding downward adjustments
The rare use of downward adjustments suggests super-
visors may also have avoided lowering manager perfor-
mances for those they wanted to retain. Downward
adjustments could not only increase the likelihood that
managers quit, but it might also decrease future manager
motivation, performance, and satisfaction, as well as in-
crease peer disapproval, and induce confrontations with
managers that supervisors would rather avoid (Bol, Keune,
Matsumura, & Shin, 2010; Poon, 2004; Tziner et al., 1996).
In Part 3 of the survey, I ask supervisors four questions
about why they avoided downward adjustments. Table 12,
Panel B, shows that the means of the 16 supervisors who
avoided downward adjustments were signi?cantly greater
on three of the four questions than the means of the 3
supervisors who gave at least one downward adjustment,
providing some support that supervisors avoided down
adjustments to preclude negative consequences for man-
agers and themselves.
Post hoc supplementary analyses
More evidence on the use of subjective adjustments
may be obtained by examining additional data.
Compression
Assuming performance is normally distributed, com-
pression (i.e., lack of variability in performance ratings)
in the performance distribution indicates that high per-
formers have not been adequately distinguished from
low performers, and may result in suboptimal manager ef-
fort levels. For example, both low and high performers may
have received the same evaluations and rewards although
high performers should have received higher evaluations
and rewards (Bol, 2011). In such a case, high performers
are discouraged and have little motivation to work hard
because they will be evaluated and rewarded similarly to
low performers (Bretz et al., 1992).
Table 13 reports the variability in pre- and post-ad-
justed employee performances. To measure variability, I
?rst computed the standard deviation of each of the 19
supervisor’s evaluations for each measure as well as the
overall evaluation. Then, I computed the mean of those
standard deviations. Table 13 reports that post-adjusted
performances have greater variability than pre-adjusted
objectively-measured performances for all measures and
for managers’ overall evaluations. The difference is signi?-
cant for measures 2 and 3, and close to signi?cant for the
overall evaluation (p = 0.12, two-tailed).
This is interesting because objective measures are com-
monly believed to be less compressed (more variable) than
subjective measures (Merchant, Stringer, &, Theivantham-
pillai, 2010; Moers, 2005; Rynes, Gerhart, & Parks, 2005).
In my setting, however, the post-adjusted objective mea-
sures were more variable and less compressed than the
pre-adjusted objective measures. Thus, supervisors’ use
of subjective adjustments, despite being predominantly
upward, helped to accomplish one of the main objectives
of the incentive plan (i.e., distinguishing manager
performances).
Qualitative analyses
The open-ended sections of the survey, which allowed
employees to write their own comments about the perfor-
mance measures and why adjustments were made, pro-
vide the ?nal bit of evidence on the use of subjective
adjustments, although manager and supervisor comments
generally differed. Regarding the measures, managers most
often commented that the measures were de?cient, and
they implicated noise and incompleteness. Representative
comments include ‘‘many things outside my control could
negatively in?uence my performance’’ and that ‘‘I am eval-
uated and expected to perform on (performance dimen-
sions unmeasured by the four objective measures)’’.
Regarding why adjustments were made, managers most
frequently commented that they ‘‘did not understand
how adjustments were used’’ and they believed supervi-
sors ‘‘corrupted the measures’’. Managers also commented
that the four measures should not have been equally-
weighted.
Supervisors also generally commented that the objec-
tive measures were de?cient. Most often, however, they
communicated that they highly valued their discretion in
making adjustments. Representative supervisor responses
suggest they made adjustments to improve individual
objective performance measurement. Such responses
include:
‘‘How do you measure Quality? Just because a person
has three ?ndings does not make it a better report than
a 1-?nding report.’’
42
Retention goals include releasing low performers as well as retaining
high performers.
Table 13
Mean variability
a
(inverse of compression) in 2006 pre- and post-adjusted
performance.
Variable 2006 Objective performance
Pre-adjusted Post-adjusted
1. PM1 0.43 0.52
2. PM2 0.48 0.85
***
3. PM3 0.48 0.73
**
4. PM4 0.35 0.50
5. Overall 0.37 0.50
PM1 = Planning measure.
PM2 = Program management measure.
PM3 = Reporting measure.
PM4 = Professional development measure.
a
To measure variability, I ?rst computed the standard deviation of
each supervisor’s evaluations (n = 19). Then, I computed the mean of
those standard deviations. I then conduct a t-test on the mean differences.
**
Statistically signi?cantly different at the 5% level (two-tailed).
***
Statistically signi?cantly different at the 1% level (two-tailed).
422 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
‘‘The (organization) uses standardized performance
measures which do not adequately capture the perfor-
mance for the individual audit managers.’’
‘‘Adjustments are an important part of the appraisal
process because the standardized performance mea-
sures do not always indicate the level of expertise or
effort from individual Audit Managers.’’
Nevertheless, a couple supervisor comments show the
wide variation, and idiosyncratic ways, in which supervi-
sors used adjustments. While one supervisor said ‘‘I largely
ignored adjustments,’’ another supervisor said he/she did
not use down adjustments, but instead ‘‘bumped people
down by not giving them (upward) adjustments.’’ In sum,
supervisor comments about the new system were gener-
ally positive; manager comments were generally negative.
Epilogue
The organization conducted an internal review of the
pay plan in 2008. The review determined that the new per-
formance management plan was accomplishing its goals
(recruiting, retaining, and rewarding high-performing
employees) as intended, but that it also unintentionally in-
creased employees’ perceptions of unfairness. Reasons for
unfairness were many; supervisor discrimination and
favoritism, and manager in?uence activities were most of-
ten cited. The report concluded that redressing fairness
perceptions was essential to the plan’s continuing success.
In 2010, new organizational leadership reinstated the for-
mer plan, citing continuing fairness concerns, and under
the premise that the system of entirely subjective mea-
sures would be fairer.
Conclusion and summary
My tests reveal a number of ?ndings. Most adjustments
(95%) are upward. Supervisors raise current, unexpectedly
low performance so that it is consistent with prior perfor-
mance when they perceive the measure of that perfor-
mance is incomplete and noisy, consistent with their
mandate of improving objective measurement. Supervisors
also raise current, unexpectedly low performance so that it
is consistent with prior performance for other reasons, and
even apparently holistically (at the manager level). Evi-
dence documents that supervisors make downward
adjustments to encourage some employees to leave the
organization, and avoid downward adjustments to pre-
clude negative consequences for managers and them-
selves. Overall, despite the organization’s best attempts
to focus all supervisors on the same purpose of improving
individual objective measurement, evidence is consistent
with supervisors using their discretion over subjective
adjustments in a variety of ways. The apparent rogue
supervisor behavior shown in this study has also existed
in other studies of subjectivity (e.g., Ittner et al., 2003).
This paper has a number of limitations. For example, the
relative importance of adjustments for measurement prop-
erties vis-à-vis ‘‘other’’ reasons remains unclear. Moreover,
I only examine a limited number of possibilities, but there
are myriad ways in which supervisors could have used
adjustments. In addition, given the manner in which the
other reasons were measured, they may be related to
adjustments by construction only. I also did not measure
the performance measures’ ‘‘true’’ de?ciencies. Instead, I
asked survey participants to judge these de?ciencies, and
because these judgments were subjective, they likely con-
tained some noise and/or bias. However, I expected super-
visors to make subjective adjustments, not based on the
true extent of the de?ciencies, which they do not know,
but based on their perceptions of measure de?ciencies.
Consistent with prior subjectivity studies, my results
suggest that there are many economic and psychological
variables affecting subjective adjustments. Indeed, there
are limitations in the theory that explains the uses of sub-
jectivity, due in part to the lack of empirical data on this
increasingly important topic (Gibbs et al., 2004; Prender-
gast, 1999). I also assume that unexpected performance
is indicated by deviations between prior year subjec-
tively-measured performance and current-year objec-
tively-measured performance, when ‘‘true’’ manager
performance may have actually varied. I mitigate this con-
cern, however, with evidence showing that supervisors’
beliefs about manager performance were remarkably con-
sistent year-on-year. The main test for downward adjust-
ments could not support or refute the theory. A data set
with a forced ranking system in place (where upward
adjustments to one manager/measure automatically im-
plies a downward adjustment to another), for example,
would, provide a more ideal test. Finally, this study is sub-
ject to the usual generalizability caveats of a ?eld study,
and thus may generalize to similar settings, such as those
with low control spans and complex work.
This paper has several implications for future research.
It would be fruitful to examine how various combinations
of subjectivity in?uence incentive system optimality. Iden-
tifying how the bene?ts and/or costs of subjectivity are im-
pacted by various monitoring systems would also be
interesting. I leave the identi?cation of these issues to fu-
ture research (Ittner & Larcker, 1998).
Acknowledgements
This paper is based on my dissertation, which I com-
pleted at Michigan State University. I thank the members
of my dissertation committee: Sue Haka, Ranjani Krishnan
(co-chair), Karen Sedatole (co-chair), and Alex von Eye. I
thank Edward Li, Christian Mastilak, K. Ramesh, Fabienne
Miller, and workshop participants at Michigan State Uni-
versity, The College of William and Mary, University of
Richmond, University of Illinois, University of Notre Dame,
Iowa State University, University of Utah, North Carolina
State University, the 2008 AAA Annual Meeting, and the
2009 Management Accounting Section Conference. I thank
the anonymous reviewers. I also appreciate the ?nancial
support provided by Michigan State University through a
Dissertation Completion Fellowship.
References
Ai, C., & Norton, E. (2003). Interaction terms in logit and probit models.
Economics Letters, 80, 123–129.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 423
Anderson, S. W., Dekker, H. C., & Sedatole, K. L. (2010). An empirical
examination of goals and performance-to-goal following the
introduction of an incentive plan with participative goal-setting.
Management Science, 56(1), 90–109.
Antle, R., & Demski, J. S. (1988). The controllability principle in
responsibility accounting. The Accounting Review, 63(4), 700–718.
Arya, A., Glover, J., Mittendorf, B., & Ye, L. (2005). On the use of customized
versus standardized performance measures. Journal of Management
Accounting Research, l7, 7–21.
Baiman, S., & Rajan, M. V. (1995). The informational advantages of
discretionary bonus schemes. The Accounting Review, 70(4), 557–579.
Baker, G. P. (1990). Pay-for-performance for middle managers: Causes
and consequences. Journal of Applied Corporate Finance, 3(3), 50–61.
Baker, G. (2000). The use of performance measures in incentive contracts.
The American Economic Review, 90(2), 415–420.
Baker, G. P., Gibbons, R., & Murphy, K. J. (1994). Subjective performance
measures in optimal incentive contracts. Quarterly Journal of
Economics, 109(4), 1125–1156.
Banker, R. D., & Datar, S. M. (1989). Sensitivity, precision, and linear
aggregation of signals for performance evaluation. Journal of
Accounting Research, 27, 21–39.
Bjerke, D., Cleveland, J., Morrison, R., & Wilson, W. (1987). Of?cer Fitness
report evaluation study. Navy Personnel Research and Development
Center report no. TR 88-4. Navy Personnel Research and Development
Center.
Bol, J. (2008). Subjectivity in compensation contracting. Journal of
Accounting Literature, 1–32.
Bol, J. (2011). The determinants and performance effects of managers’
performance evaluation biases. The Accounting Review, 86(5),
1549–1575.
Bol, J., Keune, T. M., Matsumura, E. M., & Shin, J. Y. (2010). Supervisor
discretion in target setting: An empirical investigation. The Accounting
Review, 85(6), 1861–1886.
Bol, J. C., & Smith, S. D. (2011). Spillover effects in subjective performance
evaluation: Bias and the asymmetric in?uence of controllability. The
Accounting Review, 86(4), 1213–1230.
Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S.
B. (1995). On the interchangeability of objective and subjective
measures of employee performance: A meta-analysis. Personnel
Psychology, 48(3), 587–605.
Bretz, R. D., Milkovich, G. T., & Read, W. (1992). The current state of
performance appraisal research and practice: Concerns, directions,
and implications. Journal of Management, 18(2), 321–352.
Buda, R. (1984). Effects of expectations, perceived purpose, and scale format
on performance ratings. Unpublished doctoral dissertation, Stevens
Institute of Technology, Hoboken, NJ.
Cameron, A. C., & Miller, D. L. (2011). Robust inference with clustered
data. In A. Ullah & D. E. A. Giles (Eds.), Handbook of empirical economics
and ?nance (pp. 1–28). Boca Raton: Chapman and Hall/CRC.
Datar, S., Kulp, S., & Lambert, R. (2001). Balancing performance measures.
Journal of Accounting Research, 29(1), 75–92.
Feltham, J., & Xie, J. (1994). Performance measure congruity and diversity
in multi-task principal/agent relations. The Accounting Review, 69,
429–453.
Fenner, D. B., Lerch, F. J., & Kulik, C. T. (1993). The impact of computerized
performance monitoring and prior performance knowledge on
performance evaluation. Journal of Applied Social Psychology, 23(7),
573–601.
Ferris, G. R., & Judge, T. A. (1991). Personnel/human resources
management: A political in?uence perspective. Journal of
Management, 17(2), 447–488.
Fisher, J. G., Maines, L. A., Peffer, S. A., & Sprinkle, G. B. (2005). An
experimental investigation of employer discretion in employee
performance evaluation and compensation. The Accounting Review,
80(2), 563–583.
Gibbons, R., & Murphy, K. J. (1992). Optimal incentive contracts in the
presence of career concerns: Theory and evidence. The Journal of
Political Economy, 100(3), 468–505.
Gibbs, M., Merchant, K. A., Van der Stede, W. A., & Vargus, M. E. (2004).
Determinants and effects of subjectivity in incentives. The Accounting
Review, 79(2), 409–436.
Graham, J. R., Harvey, C. R., & Rajgopal, S. (2005). The economic
implications of corporate ?nancial reporting. Journal of Accounting
and Economics, 40, 3–73.
Greene, W. (2010). Testing hypotheses about interaction terms in
nonlinear models. Economics Letters, 107, 291–296.
Heneman, R. L. (1986). The relationship between supervisory ratings and
results-oriented measures of performance: A meta-analysis. Personnel
Psychology, 39(4), 811–826.
Heslin, P. A., Latham, G. P., & VandeWalle, D. (2005). The effect of implicit
person theory on performance appraisals. Journal of Applied
Psychology, 90, 842–856.
Holmstrom, B. (1979). Moral hazard and observability. Bell Journal of
Economics, 10(1), 74–91.
Holmstrom, B., & Milgrom, P. (1991). Multitask principal-agent analyses:
Incentive contracts, asset ownership, and job design. Journal of Law,
Economics and Organization, 7(2), 24–53.
Höppe, F., & Moers, F. (2011). The choice of different types of subjectivity
in CEO annual bonus contracts. The Accounting Review, 86(6),
2023–2046.
Hopwood, A. G. (1972). An empirical study of the role of accounting data
in performance evaluation. Journal of Accounting Research, 10(Suppl.),
156–182.
Ilgen, D. R., & Favero, J. L. (1985). Limits in generalization from
psychological research to performance appraisal processes. Academy
of Management Review, 10(2), 311–321.
Ilgen, D. R., Mitchell, T. R., & Frederickson, J. W. (1981). Poor performers:
Supervisors’ and subordinates’ responses. Organizational Behavior and
Human Performance, 27, 386–410.
Ittner, C. D., & Larcker, D. F. (1998). Innovations in performance
measurement: Trends and research implications. Journal of
Management Accounting Research, 10, 205–238.
Ittner, C. D., Larcker, D. F., & Meyer, M. (2003). Subjectivity and the
weighting of performance measures: Evidence from a balanced
scorecard. The Accounting Review, 78(3), 725–758.
Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard. Boston, Mass.:
Harvard Business School Press.
Kaplan, R. S., & Norton, D. P. (2001). The strategy-focused organization.
Boston, Mass.: Harvard Business School Press.
Kravitz, D. A., & Balzer, W. K. (1992). Context effects in performance
appraisal: A methodological critique and empirical study. Journal of
Applied Psychology, 77, 24–31.
Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin,
108(3), 480–498.
Lipe, M. G., & Salterio, S. E. (2000). The balanced scorecard: Judgmental
effects of common and unique performance measures. The Accounting
Review, 75(3), 283–298.
Longenecker, C. O., Sims, H. P., & Gioia, D. A. (1987). Behind the mask: The
politic of employee appraisal. The Academy of Management Executive,
1(3), 183–193.
MacLeod, B. W., & Parent, D. (1999). Job characteristics and the form of
compensation. Research in Labor Economics, 18, 177–242.
McDonald, P. (2003). Staf?ng today’s internal audit function: Audit
executives need a realistic strategy for obtaining top talent to handle
growing demands. Internal Auditor, 60(5), 46–51.
Merchant, K. A. (1989). Rewarding results. Boston, Mass.: Harvard Business
School Press.
Merchant, K. A., Stringer, C., & Theivanthampillai, P. (2010). Relationships
between objective and subjective performance ratings. Working
paper, University of Southern California.
Meyer, M. W. (2002). Rethinking performance measurement: Beyond the
balanced scorecard. New York, NY: Cambridge University Press.
Milgrom, P. (1988). Employment contracts, in?uence activities, and
ef?cient organization design. Journal of Political Economy, 96(1),
42–60.
Milgrom, P., & Roberts, J. (1988). An economic approach to in?uence
activities in organizations. American Journal of Sociology, 94, 154–
179.
Milgrom, P., & Roberts, J. (1992). Economics, organization and management.
New Jersey: Prentice-Hall.
Moers, F. (2005). Discretion and bias in performance evaluation: The
impact of diversity and subjectivity. Accounting, Organizations and
Society, 30(1), 67–80.
Moers, F. (2006). Performance measure properties and delegation. The
Accounting Review, 81(4), 897–924.
Murphy, K. J., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985).
Effects of previous performance on evaluations of present
performance. Journal of Applied Psychology, 70(1), 72–84.
Murphy, K. R., & Cleveland, J. N. (1991). Performance appraisal: An
organizational perspective. Boston, Mass.: Allyn and Bacon.
Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance
appraisal: Social, organizational, and goal-based perspectives. Thousand
Oaks, CA: Sage.
Norton, E., Wang, H., & Ai, C. (2004). Computing interaction effects and
standard errors in logit and probit models. The STATA Journal, 4,
154–167.
Poon, J. M. L. (2004). Effects of performance appraisal politics on job
satisfaction and turnover intention. Personnel Review, 33, 322–334.
424 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Prendergast, C. (1999). The provision of incentives in ?rms. Journal of
Economic Literature, 37(1), 7–63.
Prendergast, C. (2002). The tenuous trade-off between risk and incentives.
Journal of Political Economy, 110(5), 1071–1102.
Prendergast, C., & Topel, R. H. (1993). Discretion and bias in performance
evaluation. European Economic Review, 37(2–3), 355–365.
Prendergast, C., & Topel, R. H. (1996). Favoritism in organizations. Journal
of Political Economy, 104(5), 958–978.
Rajan, M. V., & Reichelstein, S. (2009). Objective versus subjective
indicators of managerial performance. The Accounting Review, 84(1),
209–237.
Reilly, S. P., Smither, J. W., Warech, M. A., & Reilly, R. R. (1998). The
in?uence of indirect knowledge of previous performance on
ratings of present performance: The effects of job familiarity
and rater training. Journal of Business and Psychology, 12(4),
421–435.
Rynes, S. L., Gerhart, B., & Parks, L. (2005). Performance evaluation and
pay for performance. Annual Review of Psychology, 56, 571–600.
Smither, J. W., Reilly, R. R., & Buda, R. (1988). Effect of prior performance
information on ratings of present performance: Contrast versus
assimilation revisited. Journal of Applied Psychology, 73(3), 487–496.
Tayler, W. B. (2010). The balanced scorecard as a strategy-evaluation tool:
The effects of implementation involvement and a causal-chain focus.
The Accounting Review, 85(3), 1095–1117.
Thorsteinson, T. J., Breier, J., Atwell, A., Hamilton, C., & Privette, M. (2008).
Anchoring effects on performance judgments. Organizational Behavior
and Human Decision Processes, 107, 29–40.
Tziner, A., Latham, G. P., Price, B. S., & Haccoun, R. (1996). Development
and validation of a questionnaire for measuring perceived political
considerations in performance appraisal. Journal of Organizational
Behavior, 17, 179–190.
Wayne, S. J., Liden, R. C., Graf, I. K., & Ferris, G. F. (1997). The role of
upward in?uence tactics in human resource decisions. Personnel
Psychology, 50, 979–1006.
Welch, J., & Welch, S. (2006). The smarter they are. BusinessWeek, 3994,
102.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 425
doc_970862590.pdf
This field study examines whether and how supervisors’ subjective adjustments to objective
performance measures are influenced by their prior subjective evaluations of employees.
Evaluations were determined entirely subjectively in the sample internal audit
organization in 2005. In 2006, the organization introduced a pay-for-performance incentive
plan that established four objective measures of audit manager performance. Then,
knowing the challenges of objectively measuring manager performance, the organization
gave supervisors the discretion, mandate, and training to subjectively adjust each of the
objective measures when performance as indicated on the individual measures misrepresented
managers’ true performance.
Using prior-year subjectively measured performance to proxy for current-year expected
performance, empirical evidence documents that upward adjustments are more likely to
be made to unexpectedly low individual measures the more supervisors perceive deficiencies
in those objective measures.
Subjective adjustments to objective performance measures:
The in?uence of prior performance
Alexander Woods
?
The College of William and Mary, P.O. Box 8795, Mason School of Business, Williamsburg, VA 23187-8795, United States
a b s t r a c t
This ?eld study examines whether and how supervisors’ subjective adjustments to objec-
tive performance measures are in?uenced by their prior subjective evaluations of employ-
ees. Evaluations were determined entirely subjectively in the sample internal audit
organization in 2005. In 2006, the organization introduced a pay-for-performance incen-
tive plan that established four objective measures of audit manager performance. Then,
knowing the challenges of objectively measuring manager performance, the organization
gave supervisors the discretion, mandate, and training to subjectively adjust each of the
objective measures when performance as indicated on the individual measures misrepre-
sented managers’ true performance.
Using prior-year subjectively measured performance to proxy for current-year expected
performance, empirical evidence documents that upward adjustments are more likely to
be made to unexpectedly low individual measures the more supervisors perceive de?cien-
cies in those objective measures. This indicates that supervisors made adjustments to cor-
rect de?ciencies in the measures (as the organization intended). Independent of this
interaction effect, however, unexpectedly low current-year objectively-measured perfor-
mances are also more likely to be adjusted upward, which indicates supervisors also made
current performance consistent with prior performance for reasons other than to improve
individual objective measurement. Some of these other reasons are explored. The study
highlights how the impact of the implementation of a new performance measurement sys-
tem depends on the past.
Ó 2012 Elsevier Ltd. All rights reserved.
Introduction
This study examines whether and howsupervisors’ sub-
jective adjustments to objective performance measures are
in?uenced by their prior subjective evaluations of employ-
ees.
1
A large internal audit organization provides the sample
setting. In 2005, supervisors evaluated audit managers en-
tirely subjectively. In 2006, the organization introduced a
new performance measurement system that tied audit man-
ager incentives to four objective performance measures.
Then, knowing the challenges of objectively measuring man-
ager performance, the organization gave supervisors the dis-
cretion to subjectively adjust each of the individual objective
measures when performance, as indicated on the measures,
misrepresented managers’ true performance.
To understand how supervisor behavior is in?uenced by
a transition to a new performance measurement system,
this study integrates behavioral theory on ‘‘assimilation ef-
fects’’ with economic theory on performance measure-
ment. Assimilation effects would occur in my setting if
supervisors use adjustments to make current-year objec-
tive performance consistent with their prior-year subjec-
tive evaluations. If supervisors make current performance
0361-3682/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.aos.2012.06.001
?
Tel.: +757 221 2967.
E-mail address: [email protected]
1
An objective measure exists as a quantity in and of itself; in contrast,
subjective measurements are based on attitudes, beliefs, and perceptions.
These de?nitions are consistent with that of prior literature (Rajan &
Reichelstein, 2009).
Accounting, Organizations and Society 37 (2012) 403–425
Contents lists available at SciVerse ScienceDirect
Accounting, Organizations and Society
j our nal homepage: www. el sevi er. com/ l ocat e/ aos
consistent with prior performance when they perceive that
the current objective performance is de?ciently measured,
then they likely make adjustments consistent with their
mandate of improving objective performance measure-
ment. In contrast, if supervisors make current performance
consistent with prior performance for other reasons, then
they likely use subjective adjustments to pursue their
own goals. Distinguishing the reasons adjustments are
made is important because the success of the implementa-
tion of new measurement systems is likely affected by
whether supervisors make adjustments in accordance with
their mandate.
To study this issue, I combine proprietary performance
evaluation data with survey measures of supervisor per-
ceptions of de?ciencies in the objective performance
measures. I ?nd that supervisors are more likely to raise
unexpectedly low current-year objectively-measured per-
formances the more they perceive the measures of those
dimensions of performance are noisy and incomplete.
Independent of this interaction effect, however, unex-
pectedly low current-year objectively-measured perfor-
mances are also more likely to be raised, which
indicates supervisors also made current performance con-
sistent with prior performance for reasons other than to
improve individual objective measurement. Instead of
lowering unexpectedly high performances that are de?-
ciently measured, supervisors appear to use downward
adjustments to encourage the departure of certain man-
agers and avoid using downward adjustments to preclude
negative consequences for managers and themselves.
Overall, evidence is consistent with supervisors using
their discretion to both improve objective measurement
and to pursue their own goals.
This study contributes to prior literature in several
ways. First, it highlights how the effect of the implementa-
tion of a new performance measurement system depends
on the history of the prior system. Many organizations fre-
quently change their performance measurement systems,
yet relatively little is known about how such performance
measurement system implementation affects supervisor
behavior. The organization in Ittner, Larcker, and Meyer
(2003) changed its system three times from 1993 to
1998. The evidence put forth in that paper suggests that
supervisors used their discretion to prioritize ?nancial per-
formance—as had been done in the past—and thereby re-
moved the balance in the newly introduced balanced
scorecard. It was not exclusively focused, however, on
whether and how the transition to a new performance
measurement system affects supervisor behavior, as is
the current study.
Second, this study complements a recent study by Bol
and Smith (2011). Whereas it examines how objective per-
formance on one task in?uences a subsequent subjective
assessment on another task in an experimental setting,
the current study considers how prior subjective assess-
ments in?uence later assessments of objective measures
in a ?eld setting. The distinction between the reliance on
a prior subjective versus objective measure on a current
evaluation is an important one because a number of factors
related to objective and subjective measure differences
may in?uence whether and how supervisors make current
performance consistent with prior performance.
2
In addi-
tion, although Bol and Smith (2011) examine the effect of
a noisy measure on that process, I examine how several
measure properties, of which noise is one, in?uence the
process.
Third, this study extends recent work by Höppe and F.
Moers. (2011). It shows how performance measure noise
explains cross-sectional variation in the design of incentive
contracts and, in particular, when different types of subjec-
tivity might be used for CEOs. My study, in contrast, shows
how performance measure properties, including noise, af-
fect the application of subjectivity within a certain incen-
tive contract design for middle-level managers. Although
some other studies also examine the application of subjec-
tivity (e.g., Gibbs, Merchant, Van der Stede, & Vargus,
2004), they do not consider how the transition to a new
measurement system affects supervisor behavior.
Finally, prior studies in the management literature on
performance evaluation have investigated various reasons
why supervisors make current performance consistent
with prior performance, but by integrating economic the-
ory on performance measurement with behavioral theory,
this is the ?rst study to consider that assimilation effects
may result from supervisors correcting measure
de?ciencies.
The remainder of the paper is organized as follows: the
research setting section describes the research setting, the
theory section formalizes the research hypotheses, the var-
iable measurement and empirical speci?cation section de-
scribes the measurement of the variables and the empirical
speci?cations used to test the hypotheses, the results sec-
tion presents the results, and the summary and conclusion
section provides the results’ limitations and implications.
Research setting
Pay-for-performance incentive plan
This paper’s research setting comprises a large internal
audit organization. Prior to 2006, the organization pro-
vided its audit managers with annual salary increases
based on seniority. A nominal bonus was also provided
based on an overall subjective evaluation.
3
As part of a
comprehensive ‘‘performance management plan’’ aimed at
recruiting, retaining, and rewarding talented human capital,
the organization introduced a pay-for-performance incen-
tive plan in 2006. Before introducing the new plan, the orga-
nization developed four objective performance measures
based on ‘‘ideal’’ properties of objective measures that corre-
spond to academic theories of performance measurement
2
Moreover, the use of the prior objective performance information was
irrelevant in the arti?cial setting they created. Conversely, using prior
performance information can be relevant, I argue, in my setting because it
provides an expectation of what current performance on the four newly-
developed objective performance measures should be, and because super-
visors’ focus should be on trying to determine when those measures
misrepresent ‘‘true’’ performance.
3
Managers were evaluated on nine subjective measures (e.g., work
effort, communication) on a scale from 1 to 9. Thus, theoretically, overall
performance could range from 9 to 81. Actual 2005 overall subjective
evaluations ranged from 66 to 80.
404 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
[e.g., within employee control (i.e., precise), aligned with
organizational objectives (i.e., congruent), etc.]. The four
measures—Planning, Program Management, Reporting, and
Professional Development—were ones the organization be-
lieved captured entirely separate dimensions of audit man-
ager performance. The organization standardized and
equally weighted the four measures, then made managers’
annual salary increase and bonus dependent on managers’
performance on them.
The Planning measure is designed to instill ownership
over future audit work. Speci?cally, this measure requires
managers to annually identify and develop promising fu-
ture audit subjects, thereby contributing to the develop-
ment of a risk-based plan focused on improving the
organization’s processes. Success on this measure requires
managers to integrate knowledge and technical pro?ciency
to best serve future customer needs.
The Program Management measure is designed to cap-
ture the bulk of audit managing work. Program manage-
ment requires audit managers to perform preliminary
research, program design, application, summarization,
and analysis work that meets speci?ed organizational stra-
tegic plans and requirements, and generally accepted
auditing standards. Managers must identify resource
requirements prior to the audit’s application phase. They
must provide technical guidance and assistance to ?eld
auditors during this phase to facilitate timely audit com-
pletion. Managers must also prepare work papers that in-
clude evidence suf?cient to support audit objectives and
conclusions without material omissions. Managers must
pro?ciently use sophisticated computer techniques to en-
hance the audit process and analyze collected audit data.
They must also inform supervisors of all critical changes
to the audit plan, such as milestone delays, audit applica-
tion problems, and customer requests.
The Reporting measure is designed to capture the end
product of an audit manager’s work. Speci?cally, the ?nal
result of an audit is a unique audit report (or other product
such as outlines, drafts, or brie?ngs) that presents audit re-
sults, identi?es causes, and recommends corrective ac-
tions. The Reporting measure requires audit managers to
write audit reports that communicate effectively, with
minimal revisions. It also requires managers to defend
impartial audit ?ndings and recommendations to organi-
zation management of?cials, while keeping customers in-
formed and considering their views and disagreements.
The Professional Development measure is designed to
ensure that audit managers retain competency in their area
of expertise. It requires audit managers to obtain contin-
uing education and professional development training.
The organization believes that acquiring and using job-spe-
ci?c knowledge drives organizational performance. Conse-
quently, managers must complete functional training and
meet pre-speci?ed education requirements and standards.
Each performance measure is ‘‘objective’’ because per-
formance as measured against and compared to the stan-
dards actually existed as quantities before performance
was adjusted. Speci?cally, performance standards for each
measure were established prior to the measurement peri-
od. Then, at the end of the measurement period, perfor-
mance was compared to the pre-established standards
and recorded on a scale from one to ?ve. For example, for
the Planning measure, managers were expected to identify
and document two audit subjects for inclusion in the orga-
nization-wide triennial audit plan. Managers who contrib-
uted two audit subjects received a ‘‘3’’ on the Planning
measure. Contributing four audit subjects earned manag-
ers a ‘‘5’’ because their performance on this measure ex-
ceeded the established standard. Likewise, contributing
fewer than two audit subjects earned managers less than
a ‘‘3.’’ Subjectivity was likely used in the determination
of performance standards, but actual performance as com-
pared to the standards was simply recorded as a quantity
before supervisors could then make their adjustments. I
deem these measures objective because they existed as
quantities (prior to supervisors adjusting them) and could
be easily veri?ed, just as objective ?nancial accounting
measures such as pro?t exist as a quantity despite layers
of subjectivity underlying them.
Theoretically, each manager would have different
weights for each measure; this organization, however, like
many others, believed the bene?ts of standardized mea-
sures and weights exceeded the costs (Arya, Glover, Mit-
tendorf, & Ye, 2005). Nevertheless, knowing the nature
and increasing complexity of internal audit work due, for
example, to legislation such as the U.S. Sarbanes–Oxley
Act of 2002 (McDonald, 2003), and knowing the nature of
internal audit work would make objective and standard-
ized measurement dif?cult and imperfect, the organization
recognized the potential for de?ciencies in each of the
standardized, objective performance measures. As a result,
the organization authorized supervisors to adjust perfor-
mance one unit up or down to improve objectively-mea-
sured performance.
4
This resulted in an adjusted score. Performance on the
four adjusted measures was then averaged to determine
an overall ‘‘average score,’’ which, in turn, determined
the number of ‘‘shares’’ a manager would receive. These
shares (and a manager’s salary) then determined a man-
ager’s incentive pay amount.
5
For example, one manager re-
ceived values of 4, 3, 3, and 4 on the objective performance
measures. That manager’s supervisor then adjusted the mea-
sures +1, 0, 0, and 0, units, respectively, yielding adjusted
scores of 5, 3, 3, and 4. The manager’s overall average score
was 3.75 [(5 + 3 + 3 + 4)/4], which gave the manager 3
‘‘shares;’’ each share translated into 1.5% of the manager’s
base salary, which, at $74,110, awarded the manager
4
Organizational documents discuss how adjustments can bene?cially
‘‘align individual work with (the organization’s) mission and priorities’’ and
‘‘account for factors that are necessary for effective, ef?cient work
accomplishment’’ that the measures alone may not achieve. The organiza-
tion provided training to supervisors to reinforce this use of subjective
adjustments, and to preclude other uses, including making holistic
judgments (not isolating subjective adjustments on individual measures),
basing them on past performances, and trying to affect bonus payouts.
5
While the pay plan’s ‘‘template’’ permitted unequal weighting of the
four performance measures and a subjective determination of the number
of shares that could be allocated to managers based on ‘‘average scores,’’
the organization’s leaders decided to equally weight the measures and
standardize the speci?c average score ranges that equated to shares (and
then to incentive pay). Therefore, authority to subjectively weight mea-
sures and subjectively decide the number of shares to allocate to managers
was not given to supervisors.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 405
$3335 in incentive pay.
6
Details of the new pay-for-perfor-
mance plan were transparent to all employees. Managers,
for example, knew that supervisors could adjust each of
their performance measures up or down one unit.
This setting is useful for examining whether supervisors
use subjective adjustments to improve objective perfor-
mance measurement using their prior subjective evalua-
tions. The organization’s pay-for-performance plan
explicitly recognized that the objective performance mea-
sures may contain de?ciencies. The proprietary data,
which was not available in prior studies, enables me to di-
rectly measure subjective adjustments to those measures.
Moreover, the incentive system restricted supervisors to
using only subjective adjustments, rather than other forms
of subjectivity, in the evaluation and compensation pro-
cess. Finally, employees received training that reinforced
the use of adjustments to individual performance mea-
sures to improve the system of measures (e.g., the training
identi?ed ‘‘holistic’’ evaluations as improper).
Theory
The inherent complexity of managerial work often pre-
sents dif?culties in specifying and objectively measuring
the relevant dimensions, and desirable outcomes, of man-
agerial effort (Holmstrom & Milgrom, 1991; Hopwood,
1972; Meyer, 2002). Accordingly, objective measures of
managerial performance often fail to indicate a manager’s
contribution toward organizational objectives, distort their
efforts, and provide them with ineffective and/or costly
incentives (Baker, 2000; Baker, Gibbons, & Murphy, 1994;
Milgrom & Roberts, 1992; Prendergast, 1999). Although
the extensive use of objective measures suggests their ben-
e?ts outweigh their costs, these measures are nevertheless
de?cient (i.e., imperfect).
Supervisors, however, observe valuable information
about managerial performance not captured by the avail-
able objective performance measures (Baiman & Rajan,
1995; Fisher, Maines, Peffer, & Sprinkle, 2005). Thus, to
correct for imperfectly measured performance, many orga-
nizations introduce one or more forms of subjectivity into
incentive contracts (MacLeod & Parent, 1999). Subjective
adjustments to objective performance measures are used
to obtain a more accurate representation of a manager’s
contribution toward organizational objectives on particu-
lar dimensions of performance, mitigate distortions in ef-
fort, and improve incentive contracting (Merchant, 1989).
Although subjective adjustments can be bene?cial, they
also entail costs because supervisors’ incentives are often
imperfectly aligned with organizational objectives (Pren-
dergast, 2002; Prendergast & Topel, 1993; Prendergast &
Topel, 1996). Accordingly, it is unclear whether supervisors
will use their discretion over subjective adjustments to
correct for objective measure de?ciencies.
Performance measure properties
According to prior research, objective performance
measures are de?cient when they are (a) not sensitive to
managers’ actions, (b) not congruent with organizational
objectives, (c) noisy, (d) incomplete, and (e) manipulable
(Baker et al., 1994; Banker & Datar, 1989; Datar, Kulp, &
Lambert, 2001; Feltham & Xie, 1994; Holmstrom, 1979;
Holmstrom & Milgrom, 1991). Because performance mea-
sures are imperfect, they all exhibit these de?ciencies to
some degree.
Performance measure sensitivity
A performance measure is sensitive when an agent’s ac-
tion has a large expected effect on it (Banker & Datar,
1989). Measures that are high in sensitivity re?ect agent
effort; thus, they are informative about agent performance
and need no subjective adjustment. Measures that are low
in sensitivity, however, do not manifest agent effort. In
other words, insensitive measures will not always indicate
the level of effort actually put forth by agents. If a certain
(expected) effort level is to be awarded and the measure
is not particularly sensitive, one solution is to adjust the
performance target. If supervisors do not have discretion
over the target, however, another solution is to adjust the
measure, particularly when supervisors perceive the mea-
sure does not appropriately re?ect agent effort.
Performance measure congruity
A measure is congruent when an agent’s action increases
both the measure and the principal’s gross payoff (Feltham
& Xie, 1994). The extent to which a measure is incongruent
determines the extent to which that measure is de?cient.
Thus, an incongruent measure may not always indicate
when the principal’s objectives have been furthered. An
incongruent measure may increase the principal’s gross
payoff but not the score on the performance measure, or
vice versa. One solution is to add more congruent mea-
sures to an agent’s contract, so that the whole contract is
more congruent. Another solution is to adjust the perfor-
mance measure to account for manager contributions to-
ward organizational objectives that the incongruent
measure failed to recognize.
Performance measure precision
A measure exhibits precision (i.e., less noise) when var-
iation in the measure is only minimally attributable to
uncontrollable causes. The extent to which a measure does
not exhibit precision determines the extent to which it is
de?cient. Measures that are low in precision (i.e., high in
noise) are less informative about, and more likely to inac-
curately represent, managerial performance. Supervisors
can, however, correct noisy measures that have been un-
duly in?uenced by uncontrollable factors and circum-
stances by subjectively adjusting them, thereby making
them more accurately re?ect managerial performance
and improving incentive contracting. In contrast, measures
that are high in precision are already informative about
managerial performance and need no subjective
adjustment.
6
Incentive payouts ranged from 0 to $5195 and averaged $1,986 (i.e.,
two shares or 3% of salary). It is worth noting that adjustments and the
resulting overall performance evaluations were meaningful to managers
beyond the provision of one year’s incentive pay. Speci?cally, the evalu-
ations considerably in?uence managers’ promotion potential, opportunities
for training, project assignments, etc.
406 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Performance measure completeness
Completeness refers to the extent to which a measure
captures all relevant dimensions of managerial effort.
Incomplete measures can cause distortions in effort. For
example, managers may exert effort only on those perfor-
mance dimensions measured in their incentive contract
and ignore other important but unmeasured dimensions
(Holmstrom & Milgrom, 1991). A single objective perfor-
mance measure is unlikely to account for all relevant
dimensions. Therefore, multiple performance measures
are often used to evaluate managerial work (Kaplan & Nor-
ton, 1996; Kaplan & Norton, 2001; Lipe & Salterio, 2000).
Ideally, additional measures are incrementally informative
about managers’ efforts (Antle & Demski, 1988; Holm-
strom, 1979), and by expanding the set of implementable
actions, address more performance dimensions. However,
because managerial work is complex, even the use of mul-
tiple performance measures generally fails to capture all
desired and relevant managerial actions, and consequently,
incentive contracts remain incomplete.
Subjective bonuses provide one approach to dealing
with incentive contract incompleteness because they re-
ward managers for performing well on relevant but
unmeasured performance dimensions. However, just as
the entire incentive system can be incomplete, so can indi-
vidual measures in the incentive system, particularly when
they capture entirely different performance dimensions,
such as the number of inventory turns used in the same
contract with the number of days sales outstanding (Mer-
chant, 1989). Thus, by accounting for overlooked perfor-
mance-related actions at the level of individual measures,
subjective adjustments can deal with incompleteness in
each performance measure that fails to address all of the
important aspects of manager performance on that perfor-
mance dimension.
Performance measure manipulability
Some objective performance measures are more sus-
ceptible to manipulation than others. Manipulation occurs
when managers make measures indicate good perfor-
mance without actually producing real performance (Ba-
ker, 1990; Baker et al., 1994).
Measurement properties are theoretically distinct, yet
can be interrelated. For example, the sensitivity of a mea-
sure is only relevant if the measure correlates with the
organization’s payoff, as de?ned by congruity above. Like-
wise, a measure can only be congruent if it is complete
(Feltham & Xie, 1994) because an incomplete measure will
not, by construction, provide incentives for a congruent ef-
fort allocation (Holmstrom & Milgrom, 1991). My de?ni-
tion of completeness focuses on effort allocation, which
presumes a strong de?nition of congruity.
Prior-period performance
Research on ‘‘assimilation effects’’ has shown that cur-
rent evaluations are often in?uenced by prior evaluations
(Smither, Reilly, & Buda, 1988). For example, a new perfor-
mance measurement system that incorporated non-?nan-
cial measures of performance was introduced in Ittner
et al. (2003). Supervisors, however, used their discretion
to prioritize ?nancial performance as had been done in
the past. In Bol and Smith (2011), supervisors made cur-
rent subjective evaluations more consistent with the per-
formance level of a previously-observed objective
measure. Although it is known that prior performance of-
ten in?uences current performance, it is often unclear
why (Kravitz & Balzer, 1992; Murphy, Balzer, Lockhart, &
Eisenman, 1985; Thorsteinson, Breier, Atwell, Hamilton,
& Privette, 2008).
7
I argue that supervisors may make current performance
consistent with prior performance to correct measure de?-
ciencies because prior performance provides supervisors
with an expectation about what current performance
should be (Buda, 1984). Fenner, Lerch, and Kulik (1993)
?nds supervisors are more (less) certain that current per-
formance is appropriately represented when observed cur-
rent performance is (is not) consistent with expectations.
In Reilly, Smither, Warech, and Reilly (1998), evaluating
performance on unfamiliar jobs resulted in greater reliance
on prior performance information. In this paper’s research
setting, prior performance is especially likely to provide
supervisors with an expectation for evaluating current per-
formance because four newobjective measures were intro-
duced as part of the new performance measurement
system implementation. Thus, supervisors had no experi-
ence evaluating these new measures. This lack of familiar-
ity suggests that supervisors may use their prior-period
subjective performance evaluations as a basis to assess
whether current performance (as indicated on the mea-
sures) is appropriately represented.
Importantly, however, whether prior performance pro-
vides a relevant basis for current performance expectations
depends on the relative stability of that performance. Spe-
ci?cally, when performance is stable, deviations in perfor-
mance from one period to another are generally ‘‘small’’—
this means that ‘‘large’’ deviations may be useful indicators
of misrepresented performances. Thus, in circumstances
with stable performance, prior performance information
is particularly relevant as an expectation of current perfor-
mance (Heslin, Latham, & VandeWalle, 2005).
8
In contrast,
when performance is not stable, deviations from one period
to another are large and do not imply performance misrep-
resentations. In such circumstances, prior performance
information does not provide a relevant expectation for cur-
rent performance.
Hypotheses
Several assumptions underlie the ensuing hypotheses. I
assume that supervisors make adjustments in accordance
with their mandate of correcting de?ciencies. This implies
(1) symmetrical (both up and down) adjustments (2) to
individual measures (3) for the purpose of improving
objective performance measurement (4) based on measure
de?ciencies. I also base the hypotheses not on the actual
7
Some of the research assumes and/or investigate whether various
cognitive biases explain assimilation effects. Many of these experimental
studies create arti?cial settings in which the use of prior performance is
intentionally made irrelevant to the current evaluation.
8
This assumption is tested in the empirics section.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 407
de?ciencies, but on supervisor perceptions of the de?cien-
cies. If actual de?ciencies could be observed, a formulaic
system of adjustments could be used in place of the super-
visor’s assessment, which would no longer be required. In-
deed, supervisors do not observe the true, or actual, levels
of sensitivity, congruity, precision, and so on. Moreover,
supervisors’ perceptions are likely to be quite different
than the actual levels—however, the essence of subjectivity
necessitates that adjustments be based on the subjective
perceptions of supervisors.
I assume supervisors maintain ‘‘general’’ perceptions
about the properties of the performance measures because
the measures are standardized across all managers. Stan-
dardization means that performance requirements embed-
ded in each measure are identical across managers.
Nevertheless, because even a poor (good) measure has
the potential of properly (improperly) representing perfor-
mance, I also assume supervisors judge observed manager
performance for each measure using prior-year perfor-
mance. Speci?cally, the more that a manager’s observed
objective performance differs from what supervisors ex-
pect it should be (i.e., ‘‘unexpected performance’’), and
the more that supervisors perceive the objective measure
is incapable (i.e., de?cient or insensitive, incongruent,
imprecise, or incomplete) of properly indicating ‘‘true’’ per-
formance, the more likely supervisors are to conclude that
the objective performance is misrepresented, and in turn,
are more likely to adjust the measure so that it more accu-
rately represents the manager’s ‘‘true’’ performance. In
contrast, an ‘‘ideal’’ (sensitive, congruent, etc.) objective
measure indicating performance that is ‘‘as expected’’
needs no improvement. Thus, the greater the supervisor’s
perception of de?ciency in the objective measures, the
greater the likelihood of adjustments up (down) for unex-
pectedly low (high) objectively-measured manager perfor-
mances. This discussion leads to the following hypotheses:
Hypothesis 1a. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less sensitive supervisors perceive is the objective
measure of performance.
Hypothesis 1b. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less congruent supervisors perceive is the objective
measure of performance.
Hypothesis 1c. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less precise supervisors perceive is the objective
measure of performance.
Hypothesis 1d. Unexpectedly low (high) objectively-mea-
sured manager performance is more likely to be adjusted up
(down) the less complete supervisors perceive is the objective
measure of performance.
How will supervisors respond to measures they per-
ceive as manipulable? Manipulability is different than the
other properties. The simplest explanation is that manag-
ers are likely to manipulate performance upward only,
and supervisors are likely to only subjectively adjust per-
formance down to counteract the effects of manipulation.
Thus, the greater the supervisor’s perception that the
objective measures are manipulable, the greater the likeli-
hood of downward adjustments for unexpectedly high
objectively-measured manager performances. This logic
yields the following hypothesis:
Hypothesis 1e. Unexpectedly high objectively-measured
manager performance is more likely to be subjectively
adjusted down by supervisors the more manipulable super-
visors perceive is the objective measure of performance.
Using subjective adjustments for reasons ‘‘other’’ than to
improve objective performance measurement
Despite the hypotheses presented above, management
literature on performance evaluations documents that
supervisors often use evaluation discretion to pursue their
own goals (which do not always match the goals the
organization would like them to pursue) (e.g., Bjerke,
Cleveland, Morrison, & Wilson., 1987; Gibbs et al., 2004;
Longenecker, Sims, & Gioia, 1987). Thus, supervisors may
make current performance consistent with prior perfor-
mance for reasons other than to correct for measure
de?ciencies. One reason may be to in?uence manager
behavior in future periods, such as motivating future
performance, in?uencing retention, and gaining manager
support and cooperation (Bretz, Milkovich, & Read, 1992;
Ilgen, Mitchell, & Frederickson, 1981). For example:
‘‘. . . accuracy was not their primary concern. Rather,
they were much more interested in whether their rating
would be effective in maintaining or increasing the sub-
ordinate’s future level of performance’’ (Longenecker
et al., 1987, p. 187).
Supervisors may make current performance consistent
with prior performance in response to upward in?uence
activities or ingratiatory behavior by managers (Milgrom
& Roberts, 1988; Wayne, Liden, Graf, & Ferris, 1977). Spe-
ci?cally, because supervisors’ use (and non-use) of subjec-
tive adjustments has important effects on managers,
managers are incentivized to in?uence their supervisor’s
subjective assessment of their performance (Ferris & Judge,
1991). This can be costly to organizations because engag-
ing in activities designed to favorably in?uence supervi-
sory assessments diverts managers’ attention and effort
away from more organizationally productive activities
(Anderson, Dekker, & Sedatole, 2010).
Another reason supervisors may make current perfor-
manceconsistent withprior performance involves in?uence
activities of supervisors themselves. Speci?cally, supervi-
sors, in their evaluations of employees, may consider their
supervisors’ evaluation of their supervisory performance.
Because part of supervisors’ jobs includes developing their
subordinates, supervisors may use adjustments to project
the image that they are skilled leaders, increasing their po-
tential for promotion to higher levels of responsibility,
power, and control in the organization (Ilgen & Favero,
1985; Ilgen et al., 1981; Welch & Welch, 2006).
408 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Finally, supervisors may make current performance
consistent with prior performance due to favoritism,
whereby supervisors reward some managers over others
based on their personal preferences rather than on these
managers’ performance (Prendergast & Topel, 1996).
Supervisors apply favoritism in part because their incen-
tives with respect to their evaluation of managers are not
always perfectly aligned with organizational objectives.
Speci?cally, supervisors occupy just one of the multi-lay-
ered agency relationships existing in most hierarchical
organizations, and thus are not themselves full residual
claimants on manager output (Bol, 2008). As such, supervi-
sors may bene?t from applying favoritism while the orga-
nization bears the cost. Thus, supervisors may adjust
measures of a manager who is favored, even when actual
performance does not justify the adjustment.
In sum, if supervisors use adjustments to make current
performance consistent with prior performance for reasons
other than to correct de?ciencies in the objective mea-
sures, unexpectedly high (low) objectively measured per-
formance in the current period is less (more) likely to be
subjectively adjusted downward (upward). I therefore po-
sit the following hypotheses:
Hypothesis 2a. Unexpectedly low objectively measured per-
formance is more likely to be subjectively adjusted upward.
Hypothesis 2b. Unexpectedly high objectively measured per-
formance is less likely to be subjectively adjusted downward.
Summary of hypotheses
To summarize, if supervisors use adjustments to make
current performance consistent with prior performance
to improve the measurement of individual objective mea-
sures, they should be more likely to subjectively adjust
objective performance measures under certain conditions.
Supervisors should adjust objective measures upward
(downward) when managers’ performance on each of
those measures is unexpectedly low (high) and supervisors
perceive the measures are de?cient with respect to sensi-
tivity (H1a), congruity (H1b), precision (H1c), and com-
pleteness (H1d). Supervisors should also be more likely
to adjust unexpectedly-high objective measures down-
ward the more manipulable (H1e) they perceive the mea-
sure to be. If supervisors use adjustments to make current
performance consistent with prior performance for reasons
other than to correct de?ciencies in the objective mea-
sures, unexpectedly low (high) performances should be
more likely to be subjectively adjusted up (down) (Hypoth-
eses 2a and 2b, respectively).
Variable measurement and empirical speci?cation
Data collection and survey development
I collected data to test the hypotheses from three
sources. First, I collected the proprietary evaluation and
compensation data for all managers for the 2006 evalua-
tion year from the organization with oversight of the
pay-for-performance plan. Second, I collected the prior
2 years (2004 and 2005) of performance evaluation data,
as well as turnover data for the 2007 evaluation year, from
the internal audit organization. Third, I developed surveys
directed at three layer in the organization hierarchy: (1)
managers, (2) supervisors, and (3) division managers
(supervisors of the supervisors).
9
An organizational repre-
sentative helped coordinate and review the surveys, which
measured supervisor perceptions of the extent to which
the objective performance measures were de?cient and
how supervisors used adjustments.
The surveys were conducted using Internet survey
software.
10
I obtained complete survey responses from 8
of the 12 division managers (67%), 19 of the 36 supervisors
(53%), and 39 of the 130 managers (30%). All 12 divisions
were represented in the 19 supervisor responses; the mean
number of supervisor responses per division was 1.6 and
ranged from 1 to 3 (SD = 0.9). Similarly, all divisions were
adequately represented in the distribution of the entire
set of responses. Speci?cally, the mean number of re-
sponses per division was 5.5 and ranged from 4 to 9
(SD = 1.4).
Measures
An indicator variable is set to 1 (and 0 otherwise) to
measure the presence of a subjective adjustment by a gi-
ven supervisor to a particular performance measure of a gi-
ven manager.
11
Unexpected manager performance
Prior-year (2005) subjectively-measured performance
provides a baseline measure of supervisors’ expectations
of managers’ current-year (2006), pre-adjusted, objec-
tively-measured performance. Because performance across
years was evaluated on different scales, I calculate z-scores
and use the difference in standardized performance scores
across years to measure unexpected performance
(Unexp_Perf):
P
ij
ÀP
^
i
d
i
À
O
j
ÀO
^
d
ð1Þ
where P
ij
is performance on measure i for manager j, and P
^
i
and d
i
are the mean and standard deviation, respectively, of
all pre-adjusted performances on measure i in 2006. O
j
is
the overall evaluation for manager j, and O
^
and d are the
mean and standard deviation, respectively, of all overall
manager performances in 2005. Higher (lower) values in
9
I used the supervisor survey for my independent variables; the audit
and division manager surveys were used mostly for validation purposes.
10
Emails were sent to administrative assistants, who forwarded separate
emails describing the purpose of the survey to division managers,
supervisors, and managers. The emails also contained the survey links,
which, the respondents were informed, would be accessible from March 5
to March 23, 2007.
11
Nineteen supervisors completed the survey in its entirety, thus
providing me with their perceptions of the extent to which the objective
performance measures were de?cient. I have prior-year performance for 68
managers who worked for these 19 supervisors. Thus, the hypotheses use
the 272 performance measures of the 68 managers who worked for the 19
supervisors who completed the survey.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 409
Eq. (1) indicate current-year performance is higher (lower)
than prior-year performance. I divide the sample into three
groups. If Eq. (1) > 1, unexpectedly high performance (Un-
exp_High_Perf) takes the value of 1, and 0 otherwise. If
Eq. (1) < À1, unexpectedly low performance (Unexp_Low_-
Perf) takes the value of 1, and 0 otherwise. If Eq. (1) is be-
tween À1 and 1, expected performance (Exp_Perf) takes the
value of 1, and 0 otherwise. This procedure resulted in clas-
sifying 48 (18%) pre-adjusted measures as unexpectedly
high, 49 (18%) as unexpectedly low, and 175 (64%) as
expected.
Several assumptions underlie how well deviations
from the prior-year represent current-year unexpected
performance. I examine the reasonableness of these
assumptions. First, to the extent that manager perfor-
mances vary year-to-year, supervisors may not have sta-
ble expectations of manager performance (Heslin et al.,
2005). Correlating 2005 subjectively-evaluated perfor-
mance with 2004 subjectively-evaluated performance
reveals a correlation of 0.77 (p < .01, two-tailed). This
high correlation indicates that manager performance,
as subjectively assessed by supervisors during the
2 years prior to the objective system implementation,
is remarkably consistent year-on-year. As such, it pro-
vides strong support to the idea that performance is
very stable, or at least that supervisors believe it is very
stable. Accordingly, I conclude that prior-year (2005)
subjectively-measured performance appears to be a rea-
sonable baseline of supervisors’ expectation of manag-
ers’ 2006 objectively-measured performance, and may
be particularly useful for supervisors in assessing cur-
rent period performance as indicated on the objective
measures.
Second, because different supervisors may differentially
subjectively assess managers, another assumption is that
supervisor-manager pairs remain relatively constant from
2005 to 2006. Of the 39 managers surveyed (some of
whom are in my sample tests and some of whom are
not), 37 (95%) indicated they were evaluated by the same
supervisor in 2005. Thus, supervisor-manager pairs appear
to be stable in this organization. I have no reason to believe
a different level of stability in supervisor-manager pairs
exists in my sample. To the extent supervisor–manager
pairs ?uctuated across years more frequently than my sur-
vey respondents indicate, I believe it only creates noise, not
bias, in my tests.
Finally, I assume the overall subjective evaluation
supervisors gave to managers in 2005 proxies for their
expectation of managers’ 2006 performance on each of
the four objective measures. An examination of appraisal
records indicates supervisors did not distinguish across
performance dimensions when they subjectively evaluated
managers in the prior year.
12
In other words, supervisors
did not demonstrate re?ned subjective beliefs about manag-
ers on separate dimensions in prior years, but rather held a
global prior subjective impression. Thus, it is unlikely super-
visors would have subjective expectations in the current
year that are unique to each of four new objective measures
(for which they have no experience observing). Given such
evidence in this organization, as well as widespread evi-
dence in other contexts of supervisors’ tendencies to rate
managers similarly across different subjective performance
dimensions (e.g., Murphy & Cleveland, 1991, 1995), this
assumption seems reasonable but is nevertheless made
explicit.
Performance measure property perceptions
I use survey questions adapted from Moers (2006),
Gibbs et al. (2004), and Anderson et al. (2010) to measure
supervisors’ perceptions
13
of the extent of de?ciency of
each performance measure. I use a seven-point, fully-an-
chored scale to indicate the extent to which each perfor-
mance measure re?ects the following properties:
sensitivity (H1a), congruity (H1b), noise (H1c), complete-
ness (H1d), veri?ability (H1e), and manipulability (H1f).
14
Thus, each supervisor answered questions related to perfor-
mance measure properties for each of the four performance
measures.
I construct the performance measure property vari-
ables by performing common factor analysis with vari-
max rotation
15
to identify the underlying properties of
performance measures as perceived by the 19 supervisors.
I pool the data across the measures to yield a ?nal factor
analysis sample of 76 cases (19 supervisor responses  4
performance measures). The resulting factor analysis used
9 questions
16
and identi?ed 4 factors
17
: (1) sensitive/con-
gruent,
18
(2) complete, (3) noise, and (4) manipulable.
These 4 factors explained 85% of the total variance, had
factor loadings ranging from |0.76| to |0.95|, and Cron-
bach alphas from 0.84 to 0.93. See Table 1, Panel A, for
12
For example, one manager with an overall evaluation of 70 (76)
received scores of either 7 or 8 only (8 or 9 only) across the 9 performance
dimensions.
13
I use supervisor perceptions in my statistical tests because my theory is
that supervisors—not managers—make adjustments based, in part, on their
perceptions (not the actual degree) of performance measure property
de?ciencies. However, I also collected division and audit manager percep-
tions to analyze differences in perceptions across the organization hierar-
chy. While there were no differences between division manager and
supervisor perceptions, audit manager perceptions of the performance
measure properties diverged considerably from those of supervisors. In
particular, managers perceived three of the four performance measures as
being signi?cantly more de?cient than supervisors did concerning all but
one performance measure property, that property being different for each
measure.
14
Two questions (which organizational representatives requested I
include) addressed whether the performance measures captured the
quality and quantity of managerial performance. Because various factor
analyses revealed that they did not correlate highly with the other
questions to which they were a factor, I do not discuss these questions
further. I also do not further discuss two survey questions that addressed
measure veri?ability, which I included in the survey before I knew the
measures were indeed veri?able.
15
I used varimax rotation for my ?nal factor analysis to achieve
discriminant validity across the theoretically distinct constructs.
16
I dropped one question related to congruity because it had low cross-
loadings on two factors.
17
The most interpretable factor structure resulted when I retained factors
with eigenvalues >0.9 even though the conventional, though arbitrary, cut-
off value is usually 1.0.
18
Sensitivity and congruity may have loaded on the same factor because:
(1) supervisors cannot distinguish between sensitivity and congruity, (2)
sensitivity and congruity are highly correlated, and/or (3) my survey
questions failed to distinguish between sensitivity and congruity.
410 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
descriptive statistics of the questions and Panel B for the
factor analysis.
Controls
Because older supervisors generally have more expe-
rience and less career concerns than younger supervi-
sors, older supervisors may trust their subjective
assessments more and be less inhibited with their use
of adjustments than younger supervisors (Gibbons &
Murphy, 1992).
19
Using archival data, I use supervisor
age (Sup_Age) to control for differential use of adjust-
ments. In addition, because similar genders could affect
the use of subjective adjustments (Bol, 2011), I include
a binary variable that captures whether there is a gender
difference between managers and supervisors (Dif_Gen).
Table 2 reports descriptive statistics of unexpected per-
formance and control variables, and Table 3 reports a cor-
relation matrix of all independent variables in the main
analysis.
Empirical speci?cation
To test the hypotheses, I estimate two probit models in
Eq. (2) below:
Table 1
Descriptive statistics and factor analysis of the performance measure property survey items.
Panel A: Descriptive statistics of the performance measure property survey items aggregated across the 4 performance measures (n = 76)
Mean
Std.
dev. Range
Sensitive/congruent
a. If AMs perform well, it is directly re?ected in better performance on this performance measure 5.37 1.42 1–7
b. AM effort leads to better performance on this performance measure 6.07 1.02 1–7
c. The (organization’s) missions and goals are further accomplished when AMs perform well on this performance
measure
5.89 1.30 1–7
d. When AMs perform well on this performance measure, they contribute to the (organization’s) mission and goals 5.88 1.22 1–7
Cronbach’s alpha = .87
Complete
e. This performance measure does not capture important Performance measure name activities that AMs perform
a
4.34 2.04 1–7
f. This performance measure captures all the dimensions of effort that are required for AMs in this division to perform
well on this performance measure
4.01 1.97 1–7
Cronbach’s alpha = .84
Noise
g. AM performance on this performance measure is affected by unanticipated events/changes 4.79 1.90 1–7
h. AM performance on this performance measure is in?uenced by things outside of their control 4.68 1.84 1–7
Cronbach’s alpha = .93
Manipulable
i. AMs can manipulate this performance measure to ensure they meet performance goals without actually performing
well
3.47 1.69 1–7
Panel B: Common factor analysis with varimax rotation of the performance measure property survey items (all factor loadings
greater than .3 are shown, n = 76)
Items
Sensitive/
Congruent Complete Noise Manipulable
a. The (organization’s) missions and goals are further accomplished when AMs perform well on
this performance measure
0.88
b. If AMs perform well, it is directly re?ected in better performance on this performance measure 0.77
c. When AMs perform well on this performance measure, they contribute to the (organization’s)
mission and goals
0.80
d. AM effort leads to better performance on this performance measure 0.76
e. This performance measure does not capture important Performance measure name activities
that AMs perform
a
0.88
f. This performance measure captures all the dimensions of effort that are required for AMs in
this division to perform well on this performance measure
0.90
g. AM performance on this performance measure is affected by unanticipated events/changes 0.95
h. AM performance on this performance measure is in?uenced by things outside of their control 0.95
i. AMs can manipulate this performance measure to ensure they meet performance goals without
actually performing well
À0.88
Eigenvalues 3.74 2.13 1.18 0.92
AMs = audit managers.
a
This item was reverse-coded.
19
The archival measure of supervisor age is signi?cantly correlated with
the self-reported survey measure of experience (0.40, p < 0.05, one-tailed,
n = 19).
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 411
A
ijk
¼ b
0
þb
1
Unexp Perf
ij
þb
2
Insensitive Incongruent
ik
þb
3
Noise
ik
þb
4
Incomplete
ik
þb
5
Manipulable
ik
þb
6
Sup Age
k
þb
7
Dif Gen
jk
þb
8
Insensitive Incongruent à Unexp Perf
ijk
þb
9
Noise à Unexp Perf
ijk
þb
10
Incomplete
à Unexp Perf
ijk
þb
11
Manipulable à Unexp Perf
ijk
ð2Þ
in which the dependent variable in the ?rst (second) probit
model is an upward (downward) subjective adjustment (A)
to performance measure i for manager j of supervisor k,
that takes on a value of 1 if the performance measure is ad-
justed up (down) and 0 otherwise. Unexp_Perf in the up-
ward (downward) model is measured as unexpectedly
low (high) performance (i.e., Unexp_Low_Perf and Unex-
p_High_Perf). I code all performance measure properties
in negative terms, so that higher (lower) values indicate
more (less) de?ciency. That way, the coef?cients on the
interaction terms should always be positive. If supervisors
make adjustments consistent with improving objective
measurement, then upward (downward) adjustments
should be more positively related to unexpectedly low
(high) performances the more de?cient are the measures.
In both models, I use Huber–White standard errors cor-
rected for the supervisor cluster.
20
Results
Descriptive statistics
Before examining the results, Table 4 provides descrip-
tive statistics. Of the 272 measure-speci?c observations, 85
(31%) were adjusted; 81 (95%) of these adjustments were
upward. As can be seen in Table 4, Panel B, Program Man-
agement was the most often adjusted measure. The score
of ‘‘3’’ was the most frequent pre-adjusted score across
all measures, and adjustments were made to only pre-ad-
justed scores of 3 and 4. Table 4, Panel C, shows that the
use of adjustments varied greatly across supervisors.
Supervisors, on average, gave 4.5 adjustments, ranging
from 1 and 11 (SD = 2.8).
21
Table 5, Panel A, shows that adjustments varyby manager.
Of the 68 sample managers, 47 (69%) received at least one
adjustment. Adjustmentsincreased, byalmost 60%, theincen-
tive pay of 40 (85%) of the 47 managers to whomsupervisors
gave at least 1 adjustment. In addition to the meaningful im-
pact on managers’ promotability, opportunities for training,
project assignments, and so on, Panel B shows that adjust-
ments also materially impact incentive pay.
Table 6 reports the correlations between prior-year
(2005) overall subjective evaluations and current-year
(2006) pre- and post-adjustedperformances. The magnitude
of the correlation with the pre-adjusted objective measures
(0.19–0.34) is similar to the mean correlations between
objective and subjective measures reported in other studies
(Bommer, Johnson, Rich, Podsakoff, &MacKenzie, 1995; Hen-
eman, 1986). Prior-year subjective evaluations were posi-
tively correlated with current-year pre-adjusted overall
performance and the Planning, Program Management, and
Professional Development (but not the Reporting) measures;
prior-year subjective evaluations were positively correlated
with all post-adjusted measures as well as the overall post-
adjusted performance. Comparing the pre- and post-ad-
justed correlations with each other shows that the correla-
tions with post-adjusted measures are higher for the
overall evaluation and all individual measures except mea-
sure one (Planning). Thus, supervisors’ use of subjective
adjustments increased the correlation with prior-year sub-
jectively-measured performance for managers’ overall per-
formance and for the Program Management, Reporting, and
Professional Development measures (andactually decreased
it for the Planning measure).
Main analysis
The results in Panel A (B) of Table 7 are used to test the
hypotheses for upward (downward) adjustments.
22,23
Table 2
Descriptive statistics of unexpected performance and control variables.
Variable n Mean Std. dev. Range
Unexp_Low_Perf 272 0.18 0.39 0–1
Unexp_High_Perf 272 0.18 0.38 0–1
Exp_Perf 272 0.64 0.48 0–1
Unexp_Low_OV_Perf 68 0.15 0.36 0–1
Dif_Gen 68 0.51 0.50 0–1
Sup_Age 19 47.8 6.34 38–58
Unexpected performance is measured as:
P
ij
ÀP
^
i
d
i
n o
À
O
j
ÀO
^
d
n o
ð1Þ
where P
ji
is performance on measure i for manager j, and P
^
i
and d
i
is the
mean and standard deviation, respectively, of all pre-adjusted perfor-
mances on measure i in 2006. O
j
is the overall evaluation for manager j,
and O
^
and d the mean and standard deviation, respectively, of all overall
manager performances in 2005. If Eq. (1) > 1, unexpectedly high perfor-
mance (Unexp_High_Perf) takes the value of 1 and 0 otherwise. If Eq.
(1) < À1, unexpectedly low performance (Unexp_Low_Perf) takes the value
of 1 and 0 otherwise. If Eq. (1) is between À1 and 1, expected performance
(Exp_Perf) takes the value of 1, and 0 otherwise.
Unexpectedly low overall manager performance (Un_Low_OV_Perf) is
measured as:
P
j
ÀP
^
d
n o
À
O
j
ÀO
^
d
n o
ð4Þ
where P
j
is manager j’s performance, and P
^
and d is the mean and
standard deviation, respectively, of all pre-adjusted overall performances
in 2006. O
j
is the overall evaluation for manager j, and O
^
and d is the
mean and standard deviation, respectively, of all overall manager per-
formances in 2005. If Eq. (4) < À1, unexpectedly low overall performance
(Unexp_Low_OV_Perf) takes the value of 1 and 0 otherwise.
Dif_Gen is a binary variable that takes the value of 1 if there is a gender
difference between managers and supervisors, and 0 otherwise. Sup_Age
is the supervisor’s age as provided per organizational records.
20
When there are nested levels of clustering, Cameron and Miller (2011,
p. 14) advise to cluster on the more aggregate group to appropriately
correct standard errors.
21
This mean was not signi?cantly different from the mean of 3.12
(standard deviation of 2.96) for supervisor non-respondents (p > .05, two-
tailed), leading me to believe that survey respondents do not differ in
important ways from non-respondents.
22
Variance In?ation Factors (VIFs) for all models presented in this paper
indicate a lack of multicollinearity.
23
Because interaction terms in nonlinear models can be dif?cult to
interpret (Ai & Norton, 2003), I introduce interaction terms individually and
calculate z-statistics on interaction terms following Norton, Wang, and Ai
(2004).
412 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Upward adjustments
The results in Panel A of Table 7 support H1c, H1d, and
H2 (but not H1a/b) for upward adjustments. The coef?-
cient on the Unexp_Low_Perf  Insens_Incong interaction
term in Column 2 is positive but not signi?cant, and thus
H1a/b is not supported. The positive and signi?cant coef?-
cient for the Unexp_Low_Perf  Noise interaction term in
Column 3 supports H1c. Following Greene (2010), Panel
A of Fig. 1 provides graphical evidence that incorporates
the main effects of Unexp_Low_Perf and Noise, and their
interaction from the model in Column 3 of Table 7, Panel
A. The graph shows that increasing performance measure
noise from one standard deviation below to one standard
deviation above the mean increases the estimated pre-
dicted probability of upward adjustments for unexpectedly
low performances by approximately 31%, as compared
with an increase of approximately 2% for performances
that are not unexpectedly low.
24
The difference between
these increases is economically signi?cant and supports
the signi?cance of the Unexp_Low_Perf  Noise interaction
(H1c).
Table 5
Examination of how many adjustments managers generally received and
whether receiving adjustments materially affected manager pay.
Panel A: Adjustments by manager
#
Managers with 1 adjustment 20
Managers with 2 adjustments 19
Managers with 3 adjustments 5
Managers with 4 adjustments
3
Total managers with 1 or more adjustment 47
Managers with 0 adjustments
21
Total managers 68
Mean # of adjustments per manager = 1.25
Panel B: Materiality of adjustments
# of
Managers
Incentive pay
before
adjustment(s)
Incentive pay
after
adjustment(s)
Change in
incentive
pay (%)
Adjustment(s)
did change
incentive
pay
40 $62,000 $98,464 58.8
Adjustment(s)
did NOT
change
incentive
pay
7 $10,150 $10,150
0
Total Managers
with 1 or
more
Adjustments
47 $72,150 $109,515 50.5
Table 4
Descriptive information of subjective adjustment use.
Panel A: Distribution of pre-adjusted performance scores (by each
objective measure)
Pre-adjusted score
2 3 4 5 Total
PM1 1 52 12 3 68
PM2 0 45 17 6 68
PM3 1 43 20 4 68
PM4
1 47 19 1 68
Total 3 187 68 14 272
Panel B: Distribution of adjustments (by pre-adjusted performance
score and objective measure)
2 3
a
4 5 Total
PM1 0 5 4 0 9
PM2 0 26 12 0 38
PM3 0 14 7 0 21
PM4
0 13 4 0 17
Total 0 58 27 0 85
Panel C: Distribution of supervisor use of adjustments
Supervisors that made: #
0 or 1 adjustment 2
2 or 3 adjustments 8
4 or 5 adjustments 1
6 or 7 adjustments 6
8 or 9 adjustments 1
10 or 11 adjustments
1
Total # of supervisors 19
PM1 = Planning measure.
PM2 = Program Management measure.
PM3 = Reporting measure.
PM4 = Professional Development measure.
Mean # of adjustments per supervisor = 4.5.
a
All four downward adjustments (3 to PM2 and 1 to PM3) were made
to a pre-adjusted score of 3. Thus, upward (downward) adjustments were
made only to pre-adjusted scores of 3 and 4 (3).
Table 3
Correlation matrix of the independent variables used to examine whether subjective adjustments were used to improve individual objective measurement.
Variable 1 2 3 4 5 6 7
1. Unexp_Low_Perf
2. Unexp_High_Perf À0.22
3. Sensitive_Congruent À0.04 0.10
4. Noise 0.01 0.00 0.00
5. Complete À0.01 0.02 0.00 0.00
6. Manipulable 0.14 À0.07 0.00 0.00 0.00
7. Sup_age À0.01 À0.14 À0.13 À0.03 À0.07 À0.09
8. Dif_gen 0.13 0.01 À0.01 À0.03 À0.05 0.11 À0.06
Bolded Pearson correlation coef?cients are signi?cant at the 10% level (two-tailed).
24
Measure properties are factor scores and thus have a mean of 0 and
standard deviation of 1.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 413
The positive and signi?cant coef?cient on the
Unexp_Low_Perf  Incmplt interaction in Column 4 sup-
ports H1d. Panel B of Fig. 1 incorporates the main effects
of Unexp_Low_Perf and Incompleteness, and their interac-
tion from the model in Column 4 of Table 7, Panel A. The
graph shows that increasing performance measure incom-
pleteness from one standard deviation below to one stan-
dard deviation above the mean increases the estimated
predicted probability of upward adjustments for unexpect-
edly lowperformances by approximately 27%, as compared
with a decrease of approximately 9% for performances that
are not unexpectedly low. The difference between these is
economically signi?cant and supports the signi?cance of
the Unexp_Low_Perf  Incompleteness interaction (H1d).
The graphs in Panels A and B of Fig. 1 also provide evi-
dence consistent with H2a. The regression line for objec-
tively-measured performances that are unexpectedly low
lies above the line for performances that are not unexpect-
edly low. Thus, on average, unexpectedly low performance
has a positive and signi?cant main effect. To properly
interpret the positive coef?cient on Unexp_Low_Perf, fol-
lowing Greene (2010), I examine the change in probability
of adjustment by moving the indicator variable for unex-
pectedly low performance from 0 to 1 in settings of low,
medium, and high levels of de?ciency (with respect to Inc-
mplt and Noise).
25
At low levels of de?ciency, I do not ?nd a
signi?cant effect. However, at medium levels of Noise
(Incompleteness), the change to unexpectedly low perfor-
mance increases the estimated predicted probability of up-
ward adjustments by approximately 17% (20%). At high
levels of Noise (Incompleteness), the change to unexpectedly
low performance increases the estimated predicted proba-
bility of upward adjustments by approximately 32% (37%).
Thus, in support of H2a, unexpectedly low performance
has a positive and signi?cant main effect when measures
have medium or high levels of de?ciency. Overall, the results
suggest that unexpectedly low current performances were
raised to correct measure de?ciencies, as well as for other
reasons.
26
Downward adjustments
The results in Panel B of Table 7 do not support H1 or
H2 for downward adjustments. Results are generally in
the predicted direction, but are not signi?cant. For exam-
ple, for the model in Column 5 of Panel B of Table 7, the
coef?cient on the Unexp_High_Perf  Manipulability inter-
action (H1e) is positive as predicted, but insigni?cant. Fol-
lowing Greene (2010), increasing performance measure
manipulability from one standard deviation below to one
standard deviation above the mean increases the esti-
mated predicted probability of downward adjustment for
unexpectedly high performances by approximately 9%, as
compared with a decrease of approximately 4% for perfor-
mances that are not unexpectedly high. Similarly modest
effects are observed with the other interactions.
To properly interpret the coef?cients on Unexp_High_-
Perf and test H2b, following Greene (2010), I examine the
change in probability of downward adjustments by moving
the indicator variable for unexpectedly high performance
from 0 to 1 in settings of low, medium, and high levels of
de?ciency.
27
For the model in Column 5 of Panel B, at low
levels of Manipulability, there is no effect on the estimated
predicted probability of downward adjustments when the
indicator variable on unexpectedly high performance is
changed from 0 to 1. At medium (high) levels of Manipulabil-
ity, the change increases the estimated predicted probability
of downward adjustments by approximately 6% (12%). Sim-
ilar modest effects are observed with the other models in Ta-
ble 7, and thus do not support H2b. Because of the small
sample size (n = 272) and rarity of downward adjustments
(4), I conclude that traditional statistical tests lack suf?cient
power to unequivocally detect whether downward adjust-
ments were used to correct de?ciencies.
Supplemental analyses of upward adjustments
Table 8 reports additional tests of whether upward
adjustments were made to correct de?ciencies in the
objective measures.
28
Column 1 of Table 8 incorporates all
interaction terms from Table 7, Panel A, into one model. Col-
umn 2 of Table 8 then adds a new indicator variable for
unexpectedly high performances (Unexp_High_Perf) to test
for the possibility that unexpectedly high performances are
not likely to be adjusted upward, and are thus driving the re-
sults in the main analysis.
29
The results indicate that unex-
Table 6
Prior-year (2005) subjectively-measured performance correlated with
current-year (2006) pre- and post-adjusted objectively-measured
performance.
Variable 2006 Objective performance
Pre-adjusted Post-adjusted
1. PM1 0.34
***
0.22
*
2. PM2 0.31
**
0.41
***
3. PM3 0.19 0.42
***
4. PM4 0.22
*
0.32
***
5. Overall 0.26
***
0.33
***
PM1 = Planning measure.
PM2 = Program Management measure.
PM3 = Reporting measure.
PM4 = Professional Development measure.
*
Bolded Pearson correlation is statistically signi?cant at the 10% level.
**
Bolded Pearson correlation is statistically signi?cant at the 5% level.
***
Bolded Pearson correlation is statistically signi?cant at the 1% level.
25
Measures with a factor score on incomplete >0 (0) or the most incomplete (>0) measures, the coef?cient
on unexpectedly low performance remains signi?cant, and
the only signi?cant interaction (marginally) is with
favoritism.
38
Thus, supervisors make unexpectedly low current per-
formances consistent with prior performance ‘‘for other
reasons’’ only when the measures have medium or high,
but not low, de?ciency levels. Consistent with prior studies
in accounting (e.g., Tayler, 2010) and psychology (Kunda,
1990), the evidence indicates that supervisors use discre-
tion to pursue their own goals only when measures with
some de?ciency (‘‘medium’’ levels) provide them with
the leeway they need to justify giving adjustments for
the ‘‘other reasons’’ examined in this study. Measures with
low levels of de?ciency apparently prevent supervisors
from justifying adjustments for other reasons. At high lev-
els of de?ciency, adjustments to make current perfor-
mance consistent with prior performance may have been
made for still other reasons not examined in this study.
Manager level analysis
Because of the interpretation dif?culties of interaction
terms in nonlinear models, Table 11 reports the results of
an OLS regression in which the dependent variable is the
percent of adjustments made to the measures of a given
manager. Necessarily, the analysis is at the level of the
individual manager, and unexpected performance is calcu-
lated as before except at the manager level (Unexp_Lo-
w_OV_Perf).
39
Although the analysis is similar to that in
Table 10, it is important to note that it examines a subtly dif-
ferent question. Speci?cally, instead of examining whether
supervisors used adjustments to align managers’ individual
measures with their prior performance, it examines how
many adjustments supervisors used to align managers’ cur-
Table 11
OLS regressions of upward adjustments for ‘‘other’’ reasons at the manager
level (n = 68).
(1) (2)
Coeff. t-Stat Coeff. t-Stat
Intercept 0.049 0.20 À0.038 À0.15
Unexp_Low_OV_Perf [1] À0.055 À0.79 À0.129 À2.35
**
Future_Focus [2] 0.065 1.68 0.058 1.51
Mgr_In?uence [3] 0.055 1.96
*
0.054 2.03
*
Sup_In?uence [4] À0.032 À1.00 À0.033 À0.92
Favoritism [5] 0.040 1.42 0.027 1.00
Sup_age 0.006 1.08 0.008 1.35
Dif_gen À0.028 À0.42 À0.036 À0.55
[1] X [2] 0.115 2.09
**
[1] X [3] 0.162 2.79
***
[1] X [4] 0.079 1.24
[1] X [5] 0.135 3.59
***
Adj. R
2
0.18 0.20
Dependent variable = percent of performance measures adjusted upward.
Huber–White standard errors are clustered by supervisor. For variable
de?nitions, see Table 9 (Panel A) and Table 2.
*
Statistically signi?cant different at the 10% level.
**
Statistically signi?cant different at the 5% level.
***
Statistically signi?cant different at the 1% level. (one-tailed for
interaction terms and two-tailed otherwise).
38
As additional con?rmation checks, in untabulated analyses in which
the sample is restricted to the most complete (precise) measures, the main
effect of unexpectedly low performance is unrelated to upward adjust-
ments, and no interactions (except its interaction with favoritism) are
signi?cant.
39
These tests use the 68 managers who worked for the 19 supervisors.
420 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
rent overall performance with their prior performance. Hu-
ber–White standard errors, corrected for the supervisor
cluster, are used.
In Model 2 of Table 11, the coef?cients on interac-
tions between unexpectedly low performance and future
focus, manager in?uence, and favoritism are all signi?-
cant and positive. The results imply that when manag-
ers’ current-year objectively-measured performance is
unexpectedly low, supervisors make more upward
adjustments to in?uence managers’ future behavior, in
response to manager in?uence activities, and to apply
favoritism. Note, however, in Model 1 of Table 11, that
unexpectedly low overall performance was not posi-
tively related to the percentage of upward adjustments,
suggesting supervisors do not raise multiple measures to
make unexpectedly low current overall performances of
managers be consistent with prior overall performances
of managers.
40
Post hoc analyses of downward adjustments
This section reports the results of several post hoc anal-
yses that examine other reasons for the use, and non-use,
of downward adjustments.
Turnover
I analyze whether supervisors used downward adjust-
ments, at least partially, to prompt managers to leave the
organization. Of the 111 managers in the 2006 evaluation
year cycle for whom I have complete evaluation records,
a total of 10 downward adjustments were given to 7
managers.
41
Of these 111 managers, 4 left during the 2007
evaluation year. Of the 4 who left, 3 had received a down-
ward adjustment in 2006 (2 had received 1 downward
adjustment, and 1 had received 2 downward adjustments).
Table 12, Panel A, estimates a probit model of whether
the likelihood of leaving in 2007 is a function of having
performance lowered in 2006. The results suggest that one
Table 12
Post hoc analyses of other reasons for making, and avoiding, downward adjustments.
Panel A: Probit Regression of the effect of making downward adjustments on manager turnover (n = 111).
(1) (2) (3)
Left_Org
t+1
= B
0
+ B
1
One_Down_Adj
t
+ B
2
Several_Down_Adj
t
+ e
t
Intercept À2.070 À1.919 À2.070
(À7.42)
***
(À7.95)
***
(À7.42)
***
One_Down_Adj
t
1.504 1.228
(2.95)
***
(2.17)
**
Several_Down_Adj
t
1.919 0.842
(2.06)
**
(0.82)
Pseudo R
2
0.18 0.12 0.20
Panel B: Reasons for avoiding downward adjustments
Supervisors who
avoided down
adjustments
(n = 16)
Supervisors who
gave down
adjustments
(n = 3)
t-Test for mean
differences
Mean
(std.
dev.)
Range Mean
(std.
dev.)
Range p-Value
I avoided giving negative adjustments because it:
1. Might have resulted in negative consequences for AMs (e.g., demotion, no bonus,
salary freeze, etc.)
2.37 1–7 1.33 1–2 0.06
*
(2.13) (0.58)
2. Might have threatened the self-esteem of AMs 2.38 1–7 1.33 1–2 0.04
**
(1.75) (0.58)
3. Would have resulted in a written record of poor performance 2.00 1–5 1.33 1–2 0.20
(1.32) (0.58)
4. May be viewed as a failure on my part to be a good supervisor 1.81 1–4 1.00 1–1 0.01
***
(1.17) (0.00)
Dependent variable (Left_Org
t+1
) = 1 if manager left the organization during the 2007 evaluation year, and 0 otherwise. One_Down_Adj
t
= 1 if manager had
one measure adjusted downward in the 2006 evaluation year, and 0 otherwise. Several_Down_Adj
t
= 1 if manager had two or more measures adjusted
downward in the 2006 evaluation year, and 0 otherwise. Standard errors are adjusted for the supervisor cluster.
AMs = audit managers.
*
Statistically signi?cant different at the 10% level.
**
Statistically signi?cant different at the 5% level.
***
Statistically signi?cant different at the 1% level.
40
In untabulated analyses, I also tested whether supervisors used
adjustments to correct de?ciencies at the manager level (using a probit
model) and how many (using an OLS model with percent of adjustments as
the dependent variable). Empirical evidence is more supportive of super-
visors correcting de?ciencies at the level of each measure.
41
Five managers received one downward adjustment, one manager
received two downward adjustments, and one manager received three
downward adjustments.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 421
downward adjustment was suf?cient to prompt individuals
to leave the organization. Although I do not know whether
these managers were ?red, retired, found a different job,
etc. (i.e., I only know whether they left the organization dur-
ing 2007), the results are consistent with one of the over-
arching goals of the new plan—to retain the best
performers.
42
Avoiding downward adjustments
The rare use of downward adjustments suggests super-
visors may also have avoided lowering manager perfor-
mances for those they wanted to retain. Downward
adjustments could not only increase the likelihood that
managers quit, but it might also decrease future manager
motivation, performance, and satisfaction, as well as in-
crease peer disapproval, and induce confrontations with
managers that supervisors would rather avoid (Bol, Keune,
Matsumura, & Shin, 2010; Poon, 2004; Tziner et al., 1996).
In Part 3 of the survey, I ask supervisors four questions
about why they avoided downward adjustments. Table 12,
Panel B, shows that the means of the 16 supervisors who
avoided downward adjustments were signi?cantly greater
on three of the four questions than the means of the 3
supervisors who gave at least one downward adjustment,
providing some support that supervisors avoided down
adjustments to preclude negative consequences for man-
agers and themselves.
Post hoc supplementary analyses
More evidence on the use of subjective adjustments
may be obtained by examining additional data.
Compression
Assuming performance is normally distributed, com-
pression (i.e., lack of variability in performance ratings)
in the performance distribution indicates that high per-
formers have not been adequately distinguished from
low performers, and may result in suboptimal manager ef-
fort levels. For example, both low and high performers may
have received the same evaluations and rewards although
high performers should have received higher evaluations
and rewards (Bol, 2011). In such a case, high performers
are discouraged and have little motivation to work hard
because they will be evaluated and rewarded similarly to
low performers (Bretz et al., 1992).
Table 13 reports the variability in pre- and post-ad-
justed employee performances. To measure variability, I
?rst computed the standard deviation of each of the 19
supervisor’s evaluations for each measure as well as the
overall evaluation. Then, I computed the mean of those
standard deviations. Table 13 reports that post-adjusted
performances have greater variability than pre-adjusted
objectively-measured performances for all measures and
for managers’ overall evaluations. The difference is signi?-
cant for measures 2 and 3, and close to signi?cant for the
overall evaluation (p = 0.12, two-tailed).
This is interesting because objective measures are com-
monly believed to be less compressed (more variable) than
subjective measures (Merchant, Stringer, &, Theivantham-
pillai, 2010; Moers, 2005; Rynes, Gerhart, & Parks, 2005).
In my setting, however, the post-adjusted objective mea-
sures were more variable and less compressed than the
pre-adjusted objective measures. Thus, supervisors’ use
of subjective adjustments, despite being predominantly
upward, helped to accomplish one of the main objectives
of the incentive plan (i.e., distinguishing manager
performances).
Qualitative analyses
The open-ended sections of the survey, which allowed
employees to write their own comments about the perfor-
mance measures and why adjustments were made, pro-
vide the ?nal bit of evidence on the use of subjective
adjustments, although manager and supervisor comments
generally differed. Regarding the measures, managers most
often commented that the measures were de?cient, and
they implicated noise and incompleteness. Representative
comments include ‘‘many things outside my control could
negatively in?uence my performance’’ and that ‘‘I am eval-
uated and expected to perform on (performance dimen-
sions unmeasured by the four objective measures)’’.
Regarding why adjustments were made, managers most
frequently commented that they ‘‘did not understand
how adjustments were used’’ and they believed supervi-
sors ‘‘corrupted the measures’’. Managers also commented
that the four measures should not have been equally-
weighted.
Supervisors also generally commented that the objec-
tive measures were de?cient. Most often, however, they
communicated that they highly valued their discretion in
making adjustments. Representative supervisor responses
suggest they made adjustments to improve individual
objective performance measurement. Such responses
include:
‘‘How do you measure Quality? Just because a person
has three ?ndings does not make it a better report than
a 1-?nding report.’’
42
Retention goals include releasing low performers as well as retaining
high performers.
Table 13
Mean variability
a
(inverse of compression) in 2006 pre- and post-adjusted
performance.
Variable 2006 Objective performance
Pre-adjusted Post-adjusted
1. PM1 0.43 0.52
2. PM2 0.48 0.85
***
3. PM3 0.48 0.73
**
4. PM4 0.35 0.50
5. Overall 0.37 0.50
PM1 = Planning measure.
PM2 = Program management measure.
PM3 = Reporting measure.
PM4 = Professional development measure.
a
To measure variability, I ?rst computed the standard deviation of
each supervisor’s evaluations (n = 19). Then, I computed the mean of
those standard deviations. I then conduct a t-test on the mean differences.
**
Statistically signi?cantly different at the 5% level (two-tailed).
***
Statistically signi?cantly different at the 1% level (two-tailed).
422 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
‘‘The (organization) uses standardized performance
measures which do not adequately capture the perfor-
mance for the individual audit managers.’’
‘‘Adjustments are an important part of the appraisal
process because the standardized performance mea-
sures do not always indicate the level of expertise or
effort from individual Audit Managers.’’
Nevertheless, a couple supervisor comments show the
wide variation, and idiosyncratic ways, in which supervi-
sors used adjustments. While one supervisor said ‘‘I largely
ignored adjustments,’’ another supervisor said he/she did
not use down adjustments, but instead ‘‘bumped people
down by not giving them (upward) adjustments.’’ In sum,
supervisor comments about the new system were gener-
ally positive; manager comments were generally negative.
Epilogue
The organization conducted an internal review of the
pay plan in 2008. The review determined that the new per-
formance management plan was accomplishing its goals
(recruiting, retaining, and rewarding high-performing
employees) as intended, but that it also unintentionally in-
creased employees’ perceptions of unfairness. Reasons for
unfairness were many; supervisor discrimination and
favoritism, and manager in?uence activities were most of-
ten cited. The report concluded that redressing fairness
perceptions was essential to the plan’s continuing success.
In 2010, new organizational leadership reinstated the for-
mer plan, citing continuing fairness concerns, and under
the premise that the system of entirely subjective mea-
sures would be fairer.
Conclusion and summary
My tests reveal a number of ?ndings. Most adjustments
(95%) are upward. Supervisors raise current, unexpectedly
low performance so that it is consistent with prior perfor-
mance when they perceive the measure of that perfor-
mance is incomplete and noisy, consistent with their
mandate of improving objective measurement. Supervisors
also raise current, unexpectedly low performance so that it
is consistent with prior performance for other reasons, and
even apparently holistically (at the manager level). Evi-
dence documents that supervisors make downward
adjustments to encourage some employees to leave the
organization, and avoid downward adjustments to pre-
clude negative consequences for managers and them-
selves. Overall, despite the organization’s best attempts
to focus all supervisors on the same purpose of improving
individual objective measurement, evidence is consistent
with supervisors using their discretion over subjective
adjustments in a variety of ways. The apparent rogue
supervisor behavior shown in this study has also existed
in other studies of subjectivity (e.g., Ittner et al., 2003).
This paper has a number of limitations. For example, the
relative importance of adjustments for measurement prop-
erties vis-à-vis ‘‘other’’ reasons remains unclear. Moreover,
I only examine a limited number of possibilities, but there
are myriad ways in which supervisors could have used
adjustments. In addition, given the manner in which the
other reasons were measured, they may be related to
adjustments by construction only. I also did not measure
the performance measures’ ‘‘true’’ de?ciencies. Instead, I
asked survey participants to judge these de?ciencies, and
because these judgments were subjective, they likely con-
tained some noise and/or bias. However, I expected super-
visors to make subjective adjustments, not based on the
true extent of the de?ciencies, which they do not know,
but based on their perceptions of measure de?ciencies.
Consistent with prior subjectivity studies, my results
suggest that there are many economic and psychological
variables affecting subjective adjustments. Indeed, there
are limitations in the theory that explains the uses of sub-
jectivity, due in part to the lack of empirical data on this
increasingly important topic (Gibbs et al., 2004; Prender-
gast, 1999). I also assume that unexpected performance
is indicated by deviations between prior year subjec-
tively-measured performance and current-year objec-
tively-measured performance, when ‘‘true’’ manager
performance may have actually varied. I mitigate this con-
cern, however, with evidence showing that supervisors’
beliefs about manager performance were remarkably con-
sistent year-on-year. The main test for downward adjust-
ments could not support or refute the theory. A data set
with a forced ranking system in place (where upward
adjustments to one manager/measure automatically im-
plies a downward adjustment to another), for example,
would, provide a more ideal test. Finally, this study is sub-
ject to the usual generalizability caveats of a ?eld study,
and thus may generalize to similar settings, such as those
with low control spans and complex work.
This paper has several implications for future research.
It would be fruitful to examine how various combinations
of subjectivity in?uence incentive system optimality. Iden-
tifying how the bene?ts and/or costs of subjectivity are im-
pacted by various monitoring systems would also be
interesting. I leave the identi?cation of these issues to fu-
ture research (Ittner & Larcker, 1998).
Acknowledgements
This paper is based on my dissertation, which I com-
pleted at Michigan State University. I thank the members
of my dissertation committee: Sue Haka, Ranjani Krishnan
(co-chair), Karen Sedatole (co-chair), and Alex von Eye. I
thank Edward Li, Christian Mastilak, K. Ramesh, Fabienne
Miller, and workshop participants at Michigan State Uni-
versity, The College of William and Mary, University of
Richmond, University of Illinois, University of Notre Dame,
Iowa State University, University of Utah, North Carolina
State University, the 2008 AAA Annual Meeting, and the
2009 Management Accounting Section Conference. I thank
the anonymous reviewers. I also appreciate the ?nancial
support provided by Michigan State University through a
Dissertation Completion Fellowship.
References
Ai, C., & Norton, E. (2003). Interaction terms in logit and probit models.
Economics Letters, 80, 123–129.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 423
Anderson, S. W., Dekker, H. C., & Sedatole, K. L. (2010). An empirical
examination of goals and performance-to-goal following the
introduction of an incentive plan with participative goal-setting.
Management Science, 56(1), 90–109.
Antle, R., & Demski, J. S. (1988). The controllability principle in
responsibility accounting. The Accounting Review, 63(4), 700–718.
Arya, A., Glover, J., Mittendorf, B., & Ye, L. (2005). On the use of customized
versus standardized performance measures. Journal of Management
Accounting Research, l7, 7–21.
Baiman, S., & Rajan, M. V. (1995). The informational advantages of
discretionary bonus schemes. The Accounting Review, 70(4), 557–579.
Baker, G. P. (1990). Pay-for-performance for middle managers: Causes
and consequences. Journal of Applied Corporate Finance, 3(3), 50–61.
Baker, G. (2000). The use of performance measures in incentive contracts.
The American Economic Review, 90(2), 415–420.
Baker, G. P., Gibbons, R., & Murphy, K. J. (1994). Subjective performance
measures in optimal incentive contracts. Quarterly Journal of
Economics, 109(4), 1125–1156.
Banker, R. D., & Datar, S. M. (1989). Sensitivity, precision, and linear
aggregation of signals for performance evaluation. Journal of
Accounting Research, 27, 21–39.
Bjerke, D., Cleveland, J., Morrison, R., & Wilson, W. (1987). Of?cer Fitness
report evaluation study. Navy Personnel Research and Development
Center report no. TR 88-4. Navy Personnel Research and Development
Center.
Bol, J. (2008). Subjectivity in compensation contracting. Journal of
Accounting Literature, 1–32.
Bol, J. (2011). The determinants and performance effects of managers’
performance evaluation biases. The Accounting Review, 86(5),
1549–1575.
Bol, J., Keune, T. M., Matsumura, E. M., & Shin, J. Y. (2010). Supervisor
discretion in target setting: An empirical investigation. The Accounting
Review, 85(6), 1861–1886.
Bol, J. C., & Smith, S. D. (2011). Spillover effects in subjective performance
evaluation: Bias and the asymmetric in?uence of controllability. The
Accounting Review, 86(4), 1213–1230.
Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S.
B. (1995). On the interchangeability of objective and subjective
measures of employee performance: A meta-analysis. Personnel
Psychology, 48(3), 587–605.
Bretz, R. D., Milkovich, G. T., & Read, W. (1992). The current state of
performance appraisal research and practice: Concerns, directions,
and implications. Journal of Management, 18(2), 321–352.
Buda, R. (1984). Effects of expectations, perceived purpose, and scale format
on performance ratings. Unpublished doctoral dissertation, Stevens
Institute of Technology, Hoboken, NJ.
Cameron, A. C., & Miller, D. L. (2011). Robust inference with clustered
data. In A. Ullah & D. E. A. Giles (Eds.), Handbook of empirical economics
and ?nance (pp. 1–28). Boca Raton: Chapman and Hall/CRC.
Datar, S., Kulp, S., & Lambert, R. (2001). Balancing performance measures.
Journal of Accounting Research, 29(1), 75–92.
Feltham, J., & Xie, J. (1994). Performance measure congruity and diversity
in multi-task principal/agent relations. The Accounting Review, 69,
429–453.
Fenner, D. B., Lerch, F. J., & Kulik, C. T. (1993). The impact of computerized
performance monitoring and prior performance knowledge on
performance evaluation. Journal of Applied Social Psychology, 23(7),
573–601.
Ferris, G. R., & Judge, T. A. (1991). Personnel/human resources
management: A political in?uence perspective. Journal of
Management, 17(2), 447–488.
Fisher, J. G., Maines, L. A., Peffer, S. A., & Sprinkle, G. B. (2005). An
experimental investigation of employer discretion in employee
performance evaluation and compensation. The Accounting Review,
80(2), 563–583.
Gibbons, R., & Murphy, K. J. (1992). Optimal incentive contracts in the
presence of career concerns: Theory and evidence. The Journal of
Political Economy, 100(3), 468–505.
Gibbs, M., Merchant, K. A., Van der Stede, W. A., & Vargus, M. E. (2004).
Determinants and effects of subjectivity in incentives. The Accounting
Review, 79(2), 409–436.
Graham, J. R., Harvey, C. R., & Rajgopal, S. (2005). The economic
implications of corporate ?nancial reporting. Journal of Accounting
and Economics, 40, 3–73.
Greene, W. (2010). Testing hypotheses about interaction terms in
nonlinear models. Economics Letters, 107, 291–296.
Heneman, R. L. (1986). The relationship between supervisory ratings and
results-oriented measures of performance: A meta-analysis. Personnel
Psychology, 39(4), 811–826.
Heslin, P. A., Latham, G. P., & VandeWalle, D. (2005). The effect of implicit
person theory on performance appraisals. Journal of Applied
Psychology, 90, 842–856.
Holmstrom, B. (1979). Moral hazard and observability. Bell Journal of
Economics, 10(1), 74–91.
Holmstrom, B., & Milgrom, P. (1991). Multitask principal-agent analyses:
Incentive contracts, asset ownership, and job design. Journal of Law,
Economics and Organization, 7(2), 24–53.
Höppe, F., & Moers, F. (2011). The choice of different types of subjectivity
in CEO annual bonus contracts. The Accounting Review, 86(6),
2023–2046.
Hopwood, A. G. (1972). An empirical study of the role of accounting data
in performance evaluation. Journal of Accounting Research, 10(Suppl.),
156–182.
Ilgen, D. R., & Favero, J. L. (1985). Limits in generalization from
psychological research to performance appraisal processes. Academy
of Management Review, 10(2), 311–321.
Ilgen, D. R., Mitchell, T. R., & Frederickson, J. W. (1981). Poor performers:
Supervisors’ and subordinates’ responses. Organizational Behavior and
Human Performance, 27, 386–410.
Ittner, C. D., & Larcker, D. F. (1998). Innovations in performance
measurement: Trends and research implications. Journal of
Management Accounting Research, 10, 205–238.
Ittner, C. D., Larcker, D. F., & Meyer, M. (2003). Subjectivity and the
weighting of performance measures: Evidence from a balanced
scorecard. The Accounting Review, 78(3), 725–758.
Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard. Boston, Mass.:
Harvard Business School Press.
Kaplan, R. S., & Norton, D. P. (2001). The strategy-focused organization.
Boston, Mass.: Harvard Business School Press.
Kravitz, D. A., & Balzer, W. K. (1992). Context effects in performance
appraisal: A methodological critique and empirical study. Journal of
Applied Psychology, 77, 24–31.
Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin,
108(3), 480–498.
Lipe, M. G., & Salterio, S. E. (2000). The balanced scorecard: Judgmental
effects of common and unique performance measures. The Accounting
Review, 75(3), 283–298.
Longenecker, C. O., Sims, H. P., & Gioia, D. A. (1987). Behind the mask: The
politic of employee appraisal. The Academy of Management Executive,
1(3), 183–193.
MacLeod, B. W., & Parent, D. (1999). Job characteristics and the form of
compensation. Research in Labor Economics, 18, 177–242.
McDonald, P. (2003). Staf?ng today’s internal audit function: Audit
executives need a realistic strategy for obtaining top talent to handle
growing demands. Internal Auditor, 60(5), 46–51.
Merchant, K. A. (1989). Rewarding results. Boston, Mass.: Harvard Business
School Press.
Merchant, K. A., Stringer, C., & Theivanthampillai, P. (2010). Relationships
between objective and subjective performance ratings. Working
paper, University of Southern California.
Meyer, M. W. (2002). Rethinking performance measurement: Beyond the
balanced scorecard. New York, NY: Cambridge University Press.
Milgrom, P. (1988). Employment contracts, in?uence activities, and
ef?cient organization design. Journal of Political Economy, 96(1),
42–60.
Milgrom, P., & Roberts, J. (1988). An economic approach to in?uence
activities in organizations. American Journal of Sociology, 94, 154–
179.
Milgrom, P., & Roberts, J. (1992). Economics, organization and management.
New Jersey: Prentice-Hall.
Moers, F. (2005). Discretion and bias in performance evaluation: The
impact of diversity and subjectivity. Accounting, Organizations and
Society, 30(1), 67–80.
Moers, F. (2006). Performance measure properties and delegation. The
Accounting Review, 81(4), 897–924.
Murphy, K. J., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985).
Effects of previous performance on evaluations of present
performance. Journal of Applied Psychology, 70(1), 72–84.
Murphy, K. R., & Cleveland, J. N. (1991). Performance appraisal: An
organizational perspective. Boston, Mass.: Allyn and Bacon.
Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance
appraisal: Social, organizational, and goal-based perspectives. Thousand
Oaks, CA: Sage.
Norton, E., Wang, H., & Ai, C. (2004). Computing interaction effects and
standard errors in logit and probit models. The STATA Journal, 4,
154–167.
Poon, J. M. L. (2004). Effects of performance appraisal politics on job
satisfaction and turnover intention. Personnel Review, 33, 322–334.
424 A. Woods / Accounting, Organizations and Society 37 (2012) 403–425
Prendergast, C. (1999). The provision of incentives in ?rms. Journal of
Economic Literature, 37(1), 7–63.
Prendergast, C. (2002). The tenuous trade-off between risk and incentives.
Journal of Political Economy, 110(5), 1071–1102.
Prendergast, C., & Topel, R. H. (1993). Discretion and bias in performance
evaluation. European Economic Review, 37(2–3), 355–365.
Prendergast, C., & Topel, R. H. (1996). Favoritism in organizations. Journal
of Political Economy, 104(5), 958–978.
Rajan, M. V., & Reichelstein, S. (2009). Objective versus subjective
indicators of managerial performance. The Accounting Review, 84(1),
209–237.
Reilly, S. P., Smither, J. W., Warech, M. A., & Reilly, R. R. (1998). The
in?uence of indirect knowledge of previous performance on
ratings of present performance: The effects of job familiarity
and rater training. Journal of Business and Psychology, 12(4),
421–435.
Rynes, S. L., Gerhart, B., & Parks, L. (2005). Performance evaluation and
pay for performance. Annual Review of Psychology, 56, 571–600.
Smither, J. W., Reilly, R. R., & Buda, R. (1988). Effect of prior performance
information on ratings of present performance: Contrast versus
assimilation revisited. Journal of Applied Psychology, 73(3), 487–496.
Tayler, W. B. (2010). The balanced scorecard as a strategy-evaluation tool:
The effects of implementation involvement and a causal-chain focus.
The Accounting Review, 85(3), 1095–1117.
Thorsteinson, T. J., Breier, J., Atwell, A., Hamilton, C., & Privette, M. (2008).
Anchoring effects on performance judgments. Organizational Behavior
and Human Decision Processes, 107, 29–40.
Tziner, A., Latham, G. P., Price, B. S., & Haccoun, R. (1996). Development
and validation of a questionnaire for measuring perceived political
considerations in performance appraisal. Journal of Organizational
Behavior, 17, 179–190.
Wayne, S. J., Liden, R. C., Graf, I. K., & Ferris, G. F. (1997). The role of
upward in?uence tactics in human resource decisions. Personnel
Psychology, 50, 979–1006.
Welch, J., & Welch, S. (2006). The smarter they are. BusinessWeek, 3994,
102.
A. Woods / Accounting, Organizations and Society 37 (2012) 403–425 425
doc_970862590.pdf