A manipulationist view of causality in cross-sectional survey research

Description
This essay discusses issues related to establishing causal relationships in empirical survey
research. I adopt a manipulationist view of causality because it matches the context of
(management) accounting research where we are commonly interested in studying the
effects of changes. Strictly speaking, causal relationships cannot be unequivocally proven
when the researcher employs cross-sectional surveys—that is, correlation is not causation.
Notwithstanding, survey research can be fruitfully engaged to inform pertinent management
accounting topics. I discuss four ‘‘markers’’ of causality—theoretical coherence,
empirical covariation, temporal/physical separation, and internal validity—and how the
researcher can lever these to suggest compelling survey-based inferences. Of these four
markers, I particularly emphasize the first as I believe that one piece of any reasonable
observer’s considerations will be whether the proffered causal relationships are theoretically
plausible

A manipulationist view of causality in cross-sectional survey
research
q
Wim A. Van der Stede
?,1
Department of Accounting, London School of Economics, Houghton Street, London WC2A 2AE, UK
a b s t r a c t
This essay discusses issues related to establishing causal relationships in empirical survey
research. I adopt a manipulationist view of causality because it matches the context of
(management) accounting research where we are commonly interested in studying the
effects of changes. Strictly speaking, causal relationships cannot be unequivocally proven
when the researcher employs cross-sectional surveys—that is, correlation is not causation.
Notwithstanding, survey research can be fruitfully engaged to inform pertinent manage-
ment accounting topics. I discuss four ‘‘markers’’ of causality—theoretical coherence,
empirical covariation, temporal/physical separation, and internal validity—and how the
researcher can lever these to suggest compelling survey-based inferences. Of these four
markers, I particularly emphasize the ?rst as I believe that one piece of any reasonable
observer’s considerations will be whether the proffered causal relationships are theoreti-
cally plausible. Moreover, a stronger theoretical foundation also helps causal inference
by suggesting a reasonably complete set of control variables that are useful to eliminate
alternative explanations. Overall, I focus rather pragmatically on the limitations of causal
inference when using the survey method and what may be done to try and alleviate,
although not eliminate, them.
Ó 2013 Elsevier Ltd. All rights reserved.
Introduction
Research often has, intentionally or unintentionally, a
normative bent to it, suggesting that a given management
accounting
2
practice (X), say, affects some organizational
outcome such as performance (Y). When the ‘‘cause’’ has
the desired/good (undesired/bad) ‘‘effect’’, observers are
tempted to read into our articles that we ‘‘should’’ adopt
(avoid or tweak) the practice, where the practice can be par-
ticipative budgeting, subjective performance evaluations,
quality metrics in hospitals, and so on. Luft and Shields
(2003) is probably the most comprehensive review of the
management accounting literature to date that allows us
to substitute the above generically-labeled cause X and ef-
fect Y with any of a large number of variables studied in this
particular literature. Indeed, the primary goal of Luft and
Shields (2003) was to provide a critical review of the litera-
ture preoccupied with the ‘‘causes and effects of manage-
ment accounting’’ (p. 170), where prior work was selected
by them for inclusion in their review only if it dealt with
the ‘‘causes or effects of variation in management account-
ing’’ (p. 172). Hence, causes and effects of management
accounting practice seem important given the large number
of studies they found and included in their survey article
using these criteria.
When we use the term ‘‘causal’’, then, that implies in
generic notation that X must be shown to be the cause of
0361-3682/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.aos.2013.12.001
q
I am grateful to Chris Chapman for the encouragement to write this
article, as well as for his comments on earlier drafts. The ?nancial support
of CIMA is gratefully acknowledged.
?
Tel.: +44 (0)20 7955 6695.
E-mail address: [email protected]
1
Also a visiting professor at Erasmus University Rotterdam.
2
My focus on management accounting is merely practical, given my own
background in this area, but also because it is unarguably the sub?eld of
accounting where the survey method is most commonly employed (see, for
example, Hesford, Lee, Van der Stede, & Young, 2007; Van der Stede, Young,
& Chen, 2005).
Accounting, Organizations and Society 39 (2014) 567–574
Contents lists available at ScienceDirect
Accounting, Organizations and Society
j our nal homepage: www. el sevi er. com/ l ocat e/ aos
Y, or that we must be able to demonstrate that we can
change Y by manipulating X. Causality thus requires
describing how changes in the value of one (or more)
variable(s) will change the value of (an)other variable(s).
Causal relationships describe what the response of Y
would be if a certain sort of change in the value of X
were to occur. This is the manipulationist view of causality
(Woodward, 2003), which is conceptually similar to the
notion adopted by, again, Luft and Shields (2014) in their
article in the current issue of Accounting, Organizations
and Society.
Explanation in this form implies that outcomes have
causes. However, an observed correlation between out-
comes and their presumed causes does not establish causa-
tion. Strictly speaking, only experiments with
randomization can expose causal relationships. But this
should not render all other approaches short of this exper-
imental standard useless (see also, Campbell & Stanley,
1963; Cook & Campbell, 1979; Shadish, Cook, & Campbell,
2002). Indeed, although research in the physical and bio-
logical sciences may have the strongest hold on causality
in part due to their ability to set up controlled experiments,
Katz, Kahn, and Adams (1980) argue that organization re-
search has its own features, particularly that organizations
in contrast to natural phenomena are human creations
headed by people (managers, say) with the capacity and
authority to make changes.
This is particularly interesting in light of the manipu-
lationist conception of causation adopted here, as it im-
plies that the social scientist is afforded an opportunity
to study the effects of, say, these managers’ ‘‘interven-
tions’’. But how can one be sure (enough) that the pre-
sumed outcome (e.g., improved productivity) was due
to the attributed (management accounting) change when
causality cannot be unequivocally proven but instead
must be inferred, as is the case when using cross-sec-
tional surveys? As a user of this method, I have ?rst-
hand experience of how editors and reviewers often
express concern about the (internal) validity of this
method, with concerns raised about, among other things,
common-method bias
3
and establishing causal inferences.
The purpose of this article is to discuss the nature of these
concerns as well as ways through which they may be rea-
sonably alleviated.
But before doing so, I wish to underscore that an
interest in causation is not merely academic. Many have
lamented the lack of relevance of organization research
to policy or practice (e.g., Merchant, 2012; Rein, 1976).
Although the bases for the criticisms are manifold, perti-
nent for my purposes are two reasons in particular, as
quoted from Cheng and McKinley (1983, pp. 85–86,
brackets and italics added):
‘‘[First,] the single most important concern of practitio-
ners is performance: how to improve productivity,
quality of patient care, sales volume, and the like, or
how to contain costs. [Therefore,] if organization
researchers are to make their ?ndings more relevant
to practitioners, they must focus their efforts on organi-
zation performance and its determinants. A second reason
why practitioners restrict their use of organization
research is that ?ndings are not always applicable.
[Therefore,] organization research will be applied only to
the extent that the variables it deals with can be manipu-
lated in practice.’’
These two reasons indeed touch the manipulationist
notion of causality adopted here at its core: practitioners
are interested in the effects of things [determinants of per-
formance] that they can manipulate.
Hence, even if we can reasonably con?dently show
a response in Y to a change in X, but if practitioners
lack the capacity to bring about a change in X, then
such a ?nding will have little meaning for them. There-
fore, if our ?ndings were to have the greatest potential
value, they would not only scienti?cally robustly dem-
onstrate causality between variables, the causal vari-
able(s) should also be practically manipulable. This is
a daunting task indeed. Daunting enough as it is, my
essay focuses exclusively on the ?rst—the academic—
part of causal inference, although I hope somehow in
our research we will not completely ignore the rele-
vance of the second.
I start by elaborating brie?y on the key issues with
inferring causation in surveys. I then propose sensible rem-
edies (in the sense of pain relievers rather than cures), or if
not that, suggestions of care when employing surveys and
inferring and reporting survey results. I close with some
further re?ections.
Correlation is not causation
When we observe a correlation between X and Y we
cannot simply assume that Y will improve by X. There
are plenty of examples where correlation can be easily
confused with causation. For instance, studies have sug-
gested that social media users are prone to negative
emotions; that social media users experience worse emo-
tional well-being (e.g., Kross et al., 2013). If these ?nd-
ings were indicative of a causal relationship between
an individual’s social media use and emotional state,
then one would need to establish that turning to or away
from social media worsens or enhances emotional
well-being. But most of the studies examining this rela-
tionship are cross-sectional, and thus, ill-equipped if
not unable to track how an individual’s emotions change
directly with, or as a consequence of, social media use.
Indeed, because of this, the researcher cannot rule out
that perhaps people who use social media are more
prone to depression, say, in the ?rst place, which might
merely suggest that the relationship runs the other
way (thus, indicating reverse causality). Or maybe there
is a third, latent or confounding variable that affects both
social media use and one’s emotional well-being. For
3
Common-method variance or bias is a form of spurious causation
attributed to the measurement method, in this case the survey method,
where the observed relationship between an independent and dependent
variable arises from the scoring of both variables by the same respondent in
the same survey (hence, the ‘‘common variance’’) rather than from a ‘‘real’’
relationship among the constructs that the measurements of the indepen-
dent and dependent variable are purported to capture (see also Section 3.3).
568 W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574
example, people with less direct personal, real-life social
contact may be more prone to using social media and
feeling more negative.
4
The requirement that Y will improve by X—the manipu-
lationist concept of causation—is built into experimental
design; it is however much less clearly established, if at
all, through alternative methods, such as surveys as the
case in point for this essay. But this is not to suggest that
we can get wiser about causal relationships through exper-
iments only. To quote Woodward (2003: 35–36, italics
added):
‘‘There are many cases in which, for moral or practical
reasons, one must rely on non-experimental evidence
to reach causal conclusions. A plausible manipulability
theory will not deny that reliable causal inference on
the basis of non-experimental evidence is possible,
but rather suggests a speci?c way of thinking about
such inferences: we should think of them as an attempt
to determine (on the basis of other kinds of evidence)
what the results of a suitably designed hypothetical
experiment or manipulation would be without actually
carrying out this experiment. For example, for moral
and political reasons, among others, one cannot carry
out experiments in which some children are randomly
assigned to public and others to private school and their
subsequent academic careers observed. Nonetheless it
is illuminating to think of attempts to infer from non-
experimental data the effects of private schooling on
achievement as attempts to predict what the results
of such an experiment would be without actually doing
it. [. . .] An important part of the heuristic value of a
manipulability account is that by clearly spelling out what
it is that we are trying to establish when we make causal
claims (i.e., what the result of a certain hypothetical exper-
iment would be), it forces us to think in a disciplined and
speci?c way about the non-experimental evidence and
other assumptions that would be required to justify such
claims.’’
This begs the question, how can we think of observa-
tional studies in a way that mimics the ‘‘experimental
ideal’’ without actually doing an experiment? This puts
the onus as much, if not more, on the study’s underlying lo-
gic, model or theory (spelling out what it is that we are trying
to establish—and how; i.e., the overall survey research de-
sign as a hypothetical experiment) in addition to merely
‘‘putting together’’ the survey instrument (for which excel-
lent guidance such as by Dillman (1978, 1999) exists).
Explanatory survey research aims to examine causal
relationships among variables—its aim is to test theory;
that is, it starts from expectations extracted from, or build-
ing on, theory or prior work about how and why variables
are related.
5
Inferring causal relationships from survey re-
search, however, is problematic due to the nature of most
survey research designs. One reason for this problem is that
almost all survey designs are cross-sectional, meaning that
sample data are collected from a single respondent at a sin-
gle point in time. But even when a degree of temporal order
between X and Y can be established, there is still no guaran-
tee that X actually caused Y to happen, as both could be af-
fected by a third factor, meaning that there is merely a
spurious relationship between the two variables.
Hence, to be able to prudently suggest the existence of a
causal relationship, there are four things we need. First, we
need theory. Second, we need to demonstrate an associa-
tion between variables, which is the least problematic for
survey research in sharp contrast to the next two things
we need. Third, we need to demonstrate that one variable
preceded the other; that is, cause cannot follow effect. And,
?nally, we need to demonstrate that the association is not
spurious due to the in?uence of a host of other variables. I
discuss these ‘‘markers of causality’’—theoretical coher-
ence, empirical covariation, temporal (and physical) sepa-
ration, and internal validity—in the next section.
Markers of causality
Coherence
In the absence of being able to ‘‘prove’’ causality with
survey research, what can we do, then, to provide evidence
for a reasonable observer to weigh or consider? One piece
of any reasonable observer’s considerations undoubtedly
will be whether the proffered causal relationships are at
least intuitively sound if not theoretically substantiated.
It is not helpful, then, when the survey research is labeled
‘‘exploratory’’ (unless it is truly designed to be so as the
type of survey research I mentioned in footnote 4) and
the researchers claim that there is no applicable theory.
There are very few things in the social sciences about
which we know nothing or for which we cannot draw on
at least some plausible theory. Claiming a lack of any the-
ory is often seen by the reviewers as baseless, and right-
fully so.
Although it is desirable to derive hypotheses from prior
theory, I do not necessarily mean theory to be taken as
purely analytical, as the strength of theory one can draw
4
Another one of my favorite examples is where people who take out life
insurance presumably live longer (Cartwright, 1979). Reverse causality can
be reasonably ruled out here because the temporal sequence is unambig-
uous: one does not take out insurance after having lived longer. But the
in?uence of third, latent variables affecting both the propensity of owning
life insurance and longevity is ubiquitous. Indeed, it is clear that even
though owning life insurance and longevity are correlated, they are likely
both affected by a common third factor or characteristic, such as higher
income, and health consciousness. And, thus, it is unlikely that one could
increase longevity by taking out life insurance, or in the manipulationist
conception of causality, it is hard to imagine that a sensible person would
consider the decision to own life insurance as a viable ‘‘strategy’’ to live
longer. Worse, it would be plain unacceptable for life insurance salesmen to
?og life insurance policies on the basis of such an unsubstantiated, spurious
causal claim, as indeed correlation is not causation. It is much more likely
that demographic and socioeconomic explanations are at play here.
5
The other type—descriptive survey research—presents information
without trying to test a model, and thus, causation is not a concern. It is
relatively rarely featured in academic (accounting) journals with some
exceptions, such as surveys reporting executives’ motives for earnings
management (e.g., Graham, Harvey, & Rajgopal, 2005) or the adoption and
characteristics of management accounting practices like activity-based
costing (e.g., Innes, Mitchell, & Sinclair, 2000), the balanced scorecard (e.g.,
Speckbacher, Bischof, & Pfeiffer, 2003), or budgeting (Libby & Lindsay,
2010).
W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574 569
on inevitably varies from topic to topic. What I suggest in-
stead is that the researcher should at least attempt to de-
velop a plausible research model based on relevant prior
work, and in so doing, try to capture the major relation-
ships among variables (stated as hypotheses) that make
the research model interesting. In developing the research
model—that is, in establishing the major relationships
among the focal variables from prior work—one inevitably
has to trade off the complexity of the model and the taxing
limits of collecting enough pertinent data to test the rela-
tionships with a survey instrument. But the lack of a theory
or plausible research model often is the more serious prob-
lem that plagues survey research. And although this is also
true for other methods, it is possibly disproportionally so
for survey research due to its weaknesses to establish
internal validity as I will discuss across the three following
Sections 3.2–3.4. To put bluntly, weak theory combined
with a method that is naturally susceptible to weaknesses
in establishing causality leaves little to be liked about a
study. The opposite is probably also true, notably that
reviewers or readers can be rather forgiving about reason-
able and/or practical limitations of the method if the re-
sults that can be inferred from it are ‘‘consistent with’’
coherent theoretical arguments about the relationships
among the studied variables.
Aside from this taste issue, the research model also of-
ten is the source of serious problems that will become
apparent later in the study, particularly the omission of
plausible confounding variables, which I will also discuss
further below, particularly in Section 3.4. If the researcher
argues that there is a relationship between X and Y, but if
there is reason to believe that a confounding variable
causes both X and Y to move together, then the model is
misspeci?ed. Even with the best of survey instruments,
the blunder was to omit the confounding variable(s). When
this is pointed out later by the reviewers, the very nature of
surveys make such mistakes hard to recover from given
that the researcher cannot normally go back to collect
the data from the same respondents under the same cir-
cumstances. Survey research is one-shot and sunk once
done, and therefore, high risk from this angle. This is not
to discourage researchers from using the very valuable re-
search method that survey research is or can be, but rather
to drive home the point that it requires careful theorizing
or spelling out in advance what it is that we are trying to
establish. The more careful the research model is thought
out, the better the suggested causal relationships will be
substantiated and the lower the risk that plausible corre-
lated variables will be irreparably omitted from incep-
tion—in other words, the more likely that the research
will be credibly coherent and reasonably inclusive.
Covariation
If the survey research is sound, the absence of a correla-
tion is evidence that a causal relationship does not exist;
the reverse, however, is problematic. Put differently, sur-
vey evidence is better at ‘‘ruling out’’ than at ‘‘ruling in’’
causal relationships. But there is an irony to this due to
published research being generally predisposed to
supporting theory-based expectations rather than seeking
to rule them out. I return to this later (in Section 3.3).
Assume that the survey researcher ?nds a cross-sec-
tional positive association between ?nancial target dif?-
culty and the incidence of performance manipulation at
the organizational entity level (e.g., in division-level bud-
gets). Although it is perfectly reasonable to argue that
managers in entities that face more dif?cult targets are
more likely to resort to performance manipulation to meet
those targets (e.g., Merchant, 1990), and thus to argue that
pressure causes manipulation, the survey information is
not causal; it is only correlational. Indeed, we cannot know
from the correlation whether an increase (decrease) in
?nancial pressure ‘‘causes’’ managers to engage more (less)
in performance manipulation. Moreover, it is not entirely
implausible that the causal connection could go the other
way; i.e., performance manipulation could cause an in-
crease in ?nancial pressure (because the manipulations
bring future performance forward, because superiors
anticipate manipulative behaviors, and so on). The key
point, though, is that either causal direction results in the
same observed correlation; that is, the two causal models
are observationally equivalent to each other in a non-
experimental study. How then to proceed from correlation
even though only correlations are observable in a survey
study?
To be frank, there is little that can be done to substan-
tially enhance the con?dence in causal statements both
practically and at reasonable cost, beyond—again—having
good, compelling theory (see Section 3.1). That said, every-
thing that can be reasonably done to alleviate reservations
about the validity and direction of causation should be
done.
For example, the researcher could try to establish retro-
active association (instead of merely contemporaneous or
concomitant association) by asking the survey respondent
about ‘‘changes’’ in ?nancial pressure and ‘‘changes’’ in
the propensity to manipulate performance. It goes without
saying that, then, the question(s) about change in the ?rst
(?nancial pressure) should be separated in the survey from
the question(s) about change in the other (performance
manipulation). Stating the hypothesis as a part of the ques-
tion does not allow one to draw a relationship among vari-
ables; instead, rather than improving on contemporaneous
correlation, it makes it worse by establishing an embedded
correlation. For example, if a survey can collect informa-
tion about the perceived ?nancial pressure for some peri-
ods and also, as disconnected as possible from this in the
survey, information about the propensity to manipulate
performance over the same periods, then the researcher
has a reasonable basis to suggest a causal relationship that
is consistent with a compelling theory to that effect. By
cleverly exploiting lags, the researcher could even try to
alleviate concerns about reverse causality (i.e., that it is
an increase in ?nancial pressure that is associated with a
subsequent increase in performance manipulation).
All this is inevitably hard to execute in survey research
where measurement error probably increases with the de-
gree to which questions draw on respondents’ memory to
go back several periods and/or where survey length is se-
verely limited. That said, whatever can be reasonably done
570 W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574
to mimic the ‘‘experimental ideal’’ without actually doing
an experiment, should be done. Even only a limited version
of trying to establish retroactive correlation (e.g., only for
the two most recent quarters in the example above) beats
a question that essentially incorporates the hypothesis
(e.g., a question that asks respondents to indicate their de-
gree of agreement with a statement positing that ?nancial
pressure encourages performance manipulation). Put dif-
ferently, while correlation is not causation, correlation
among variables from which as much common variance
as is reasonable can be removed will make it less suspect,
and hence, more credible for readers to consider in support
of theoretically-argued causal relationships.
Separation
As my discussion above about trying to establish retro-
active association suggests, survey research is often consid-
ered to be synonymous with a cross-sectional design in
which all data are collected at once. Needless to say, a
stronger design for survey research would involve a longi-
tudinal approach in which data are collected at more than
one point in time, ideally also containing a control group
(i.e., a group that did not experience the effect of the pre-
sumed causal variable). The advantage of this design is in
the evidence it provides towards causality (e.g., Mitchell
& James, 2001). Pure cross-sectional research is the weak-
est design for drawing causal inferences because the re-
searcher can only show that two variables are correlated
at one point in time. Except perhaps through trying to
establish some retroactive association within an otherwise
cross-sectional survey, the researcher can, at best, only as-
sert that one variable causes a change in the other in the
expected direction.
A longitudinal design provides more credible evidence
of causality as one can measure changes in variables over
time and because, simply put, an effect cannot precede
its cause. If one introduces a balanced scorecard and man-
agerial long-term orientation increases after implementa-
tion, then that is better evidence that balanced
scorecards contribute to long-term focus than can be in-
ferred from observing that both are contemporaneously
correlated.
No matter how attractive a longitudinal approach may
seem, however, one should not forget that longitudinal de-
signs are appropriate only for studying phenomena that
change. Although this seems obvious, this is good news
and bad news. To start with the positive, this is the context
in which I adopted the manipulationist conception of cau-
sality for this article. Moreover, using Katz et al.’s (1980)
phraseology, the reader of this article is likely a researcher
who is interested in studying phenomena that are ‘‘human
creations headed by people with the capacity and author-
ity to make changes.’’ As (management) accountants, we
are interested indeed to examine how an assortment of
organizational practices such as, say, budgeting, perfor-
mance measurement, incentive compensation, and risk
management systems have their claimed effects; that is,
those intended by those implementing or tweaking the
systems.
The bad news is that organizational and economic
changes are hard to pin down with respect to their effects.
In stylized terms, though, the idea is to measure the key
predictors at t
0
and a given period later at t
1
(one year later,
say), we come back to measure the purported effect.
Hence, rather than at a single point in time like in a
cross-sectional survey, we collect data at two (or more)
points over time in the same organization. We are thus try-
ing to capture prospective associations rather than recreat-
ing or recalling retrospective associations. Although this
sounds great, doing so is highly problematic if not outright
impractical in most cases (for example, see Rind?eisch,
Malter, Ganesan, & Moorman, 2008).
One of the problems is to determine the right time lag
over which the effect is expected to have taken place. In
the example above, when exactly is a balanced scorecard
implementation expected to have had its full effect? Mea-
sure too soon, and the effect may not have happened yet;
wait too long and the effect may have decayed. The time
lags also are almost assuredly going to vary for each orga-
nization that has implemented the system. And, it would
be hard to ?nd a sample of ?rms that have implemented
a balanced scorecard around roughly the same window
t
0
.
6
All this would cause us to have a lot, if not an impractical
and unmanageable number, of different temporal periods in
one study. Moreover, the longer the time lag, the higher the
chance that other confounds arise due to intervening events
(e.g., there was a change of management in some ?rms that
implemented the balanced scorecard). There is also likely
going to be a decrease in sample size due to respondent drop
out, which may not be unbiased as perhaps those ?rms or
managers with bad implementation experiences may be
more likely to stop their cooperation in the study. All told,
then, longitudinal survey research appears to be a great rem-
edy to advocate but a hard one to implement. Longitudinal
surveys also require more resources (time and money).
Given these considerable hurdles to collecting good
longitudinal data by survey, let’s perhaps not forget that
in the end the degree to which the independent and
dependent variables are able to evince causality is still
theory dependent. To quote Rind?eisch et al. (2008, p.
264), ‘‘given coherence’s reliance on theory (rather than
data collection), longitudinal data will not necessarily
provide stronger evidence of coherence than cross-sec-
tional data.’’
That said, however, there is a likely bias in published
academic work stemming from empirically supporting
theory; that is, ?ndings that contradict theory face an
uphill gradient for publication. Therein lies the rub. In-
deed, as long as the ?ndings support the theory, incremen-
tally-robust longitudinal data may not have a
tremendous edge over reasonably-robust cross-sectional
6
It is fair to note here that for several topics studied in management
accounting research there are relatively few exogenous changes that affect
a large number of ?rms at the same time, except perhaps in the area of
compensation or, interestingly, pertaining to public organizations like
hospitals. However, where there are sector-wide changes, there are usually
also disclosures, which can obviate the need to rely on surveys as the
chosen method of data collection and instead permit the use of archival
panel data, say, which may allow to mitigate issues of causality that are less
or not feasible with survey data.
W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574 571
data. But when they do not—that is, when the results are
contrary to expectations—the stronger the research de-
sign, the greater the con?dence it will provide to editors
and reviewers to accept the non-results as valid. In other
words, although near-impracticable, results obtained
from a longitudinal design with the presence of the con-
trol group would be almost so hard to dismiss as invalid
to reject the study even though the ?ndings do not sup-
port an accepted theory. This is an interesting tension, as
we may stand to learn as much from refuting theory
than merely always supporting well-established
(although hopefully incrementally-advanced) theory, but
the demands on the method for tolerating refutation
may be greater than for supporting theory. That is, when
we obtain non-results, these are more likely to be dis-
missed as due to some method-related issue, even
though, ironically perhaps, the survey method is, all else
equal, strictly-speaking conceptually stronger at ruling
out than ruling in causation.
Given the impediments of longitudinal survey designs,
alternative data-collection approaches to try and
strengthen causal claims are to gather data from multi-
ple sources or multiple respondents rather than over
multiple periods. This achieves spatial rather than (or
in addition to) temporal separation between the conjec-
tured cause and its effect. It is particularly helpful to
overcome what I described in the prior subsection as
embedded covariation (e.g., to avoid that those involved
in implementing balanced scorecards may be excited
(or frustrated) enough about it such that it clouds their
perceptions of the balanced scorecard’s positive (or neg-
ative) effects). Gathering survey data for predictors and
archival data for effects may be viable, and surely recom-
mendable if it is, for research that examines outcomes
that can be objectively obtained from publicly-available
sources (e.g., ?nancial performance, warranty claims).
However desirable this may be, let’s not forget though
that one of the main reasons why surveys are commonly
employed is precisely because the constructs of interest
are perceptual (as opposed to objective) and/or good
proxies for them are not available in public or even pro-
prietary archival databases.
Yet another way to obtain some separation between
the independent and dependent variables to strengthen
claims of causality is to survey multiple informants by
collecting, say, data about predictors from one respon-
dent and data about effects from another respondent.
This is certainly more doable than employing a longitu-
dinal survey, although it is likely to restrict both sample
selection (e.g., it can only be done in organizations that
have multiple respondents in a given role, thus typically
larger ones) and sample size (i.e., it is likely to result in
lower response rates due to incomplete pairs of respon-
dents). Despite these dif?culties, the idea is that multi-
ple, independent sources of data enhance the
con?dence that the results of survey research do not
arise from an issue with the data collection method, such
as common method bias.
But, even in the case of the presumably strongest
from of longitudinal survey research, we still come up
short of ‘‘proving’’ causality. This is because there is no
guarantee that factors associated with the outcome actu-
ally caused the outcome to happen. Both the predictor(s)
and the outcome still could have been caused by a third
factor. Applied to the same example, even when having
established temporal sequence, it could still be that there
is ‘‘something else’’ about ?rms that implement balanced
scorecards (e.g., balanced scorecards are implemented
when ?rms face performance dif?culties, or balanced
scorecard implementations tend to go hand in hand with
changes in decision rights, and it is these that are affect-
ing managers’ long-term orientation). The experimental
way to handle this is with a control group, where com-
paring the experimental and control groups allows the
researcher to isolate, in this example, the impact of the
balanced scorecard implementation. Needless to say that
attempting this with a survey is merely wishful thinking.
I discuss alternative cover for this next in Section 3.4.
Validity
To go back to the earlier example of whether ?nancial
pressure in a budgeting context, say, causes managers to
manipulate performance, the relationship could altogether
be dismissed as spurious because the correlation between
?nancial pressure and performance manipulation is not
due to any causal path going from one of those two vari-
ables to the other; instead it arises from a third variable
that affects both the independent and dependent variable,
say, ?nancial distress or environmental turbulence. The
problem of causal inference from association arises here
from under-speci?cation; that is, many different causal
paths can explain the association between the two focal
variables, including one from X to Y, one from Y to X, but
also one from a third variable Z to X and one from Z to Y
(Hausman, 1998). Section 3.3 was mostly about trying to
rule out the second, reverse path.
Addressing validity concerns due to a third confounding
variable requires that the researcher attempts to rule out
the possibility of any of a reasonable number of plausible
alternative explanations for the variation in the dependent
variable. This can be done with statistical controls to some
extent, provided of course, as I said above, that the re-
searcher has identi?ed all of a reasonable set of plausible
alternative explanatory or control variables before collect-
ing the data. This puts the onus back on the research mod-
el, as analysis rarely can fully substitute for theory.
Therefore, and once again, because all cues to causality
are fallible, perhaps the most credible basis for causal
claims is rooted in the degree to which the results conform
to theory. What’s more, and what ties all of the above sub-
sections together, is that a stronger theoretical foundation
also helps causal inference by (i) guiding construct selec-
tion; (ii) specifying the most plausible direction of the cau-
sal path; and (iii) suggesting a reasonably complete set of
control variables that are useful to eliminate alternative
explanations, which implies that ‘‘researchers need to em-
ploy a combination of strong theory, careful survey design,
and appropriate statistical tools’’ (Rind?eisch et al., 2008,
p. 276).
572 W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574
Concluding thoughts
Inferring causation from detected empirical relations
is challenging for non-experimental research of any
type, and especially for single-respondent, single-period,
cross-sectional surveys. But rather than fret about the
dif?culties or being philosophical about causation, I
took a different tack in this essay, perhaps a pragmatic
one, essentially arguing that if we can establish a com-
pelling theoretical causal model (Section 3.1); then ?nd
an association between the focal variables (Section 3.2);
maintain that one variable, the cause, logically precedes
the other, the effect (Section 3.3); and mitigate con-
founding effects (Section 3.4), we may reasonably con-
?dently, although never assuredly, argue for a causal
relationship.
Even so, the prudent approach is to present to the read-
er the evidence for the model in an unbiased way and dis-
cuss its implications anticipating that the reader is willing
to accept the evidence. Language matters in this respect.
Saying that there is evidence to support the model instead
of to prove it makes more than just a subtle difference. Sim-
ilarly, using associated with is preferred to saying causing,
and so on. In the end, positioning the research in light of
the relevant theory, deciding on a suitable research meth-
od, properly analyzing the data, and judiciously interpret-
ing the results are all critical for any study’s robustness,
of which causal inference is but one of the issues. If the
readers feel the study is well-executed along these lines,
they are more likely to be willing to accept its implications
based on evidence presented in the study and related to
other research.
Given the importance of ‘‘theory’’ in this essay as essen-
tially the indispensable basis to make any reasonable cau-
sal inferences, I hasten to add that I am not narrow-minded
about the meaning of theory. I do, for example, count com-
pelling ‘‘background knowledge’’—such as that obtained
from good ?eld research—as theory. Not only is theory
development quintessential to any ?eld, method develop-
ment is, too. I am thinking about complementarities across
research methods particularly. Experimental designs are
much stronger at demonstrating causal mechanisms. Per-
haps we can take variables that appear promising in survey
research (even though they may not ‘‘prove’’ causality) and
apply them in experiments. When large-sample archival
research suggests possible intervening effects that were
not part of the a priori theory, perhaps subsequent survey
research can provide some clues as to whether those ef-
fects are robust against a causal model that expressly in-
cludes these effects before the survey data are collected.
And so on.
I also alluded to the idea that the difference between
correlation and causation is not just academic but also of
practical importance. After all, practitioners, the ultimate
users of academic knowledge, are likely keen to under-
stand what it is they can change with which effect. But
there is a rub to this, of course, as practitioners can easily
read more in the study than it strictly can speak to given
the numerous imponderables that affect human behavior
and organizational practices. But when it comes to a choice
between a well-designed study of an interesting problem
that produces results that require some careful inferences
vs. an equally well-designed study on an already well-
trodden issue which produces perhaps less equivocal and
more statistically signi?cant results, I think I would much
rather see the ?rst.
Survey research will inevitably remain a commonly-ap-
plied method for several reasons, not least because it al-
lows to measure variables about which no secondary
data are available and/or assess behaviors that are not
otherwise readily observable. And it may become even
more feasible and cost-effective to do so with new technol-
ogies such as through apps on smart phones, which may
allow tracking information less obtrusively and both more
instantly and repeatedly, thus providing the means to pos-
sibly achieve some of the longitudinal standards that can
strengthen causal inference. Here is to applying these ad-
vances to interesting questions that are theoretically com-
pelling and practically relevant.
References
Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental
designs for research. Chicago, IL: Rand-McNally.
Cartwright, N. (1979). Casual laws and effective strategies. Nous, 13(4),
419–437.
Cheng, J., & McKinley, W. (1983). Toward an integration of organization
research and practice. Administrative Science Quarterly, 28(1),
85–100.
Cook, T., & Campbell, D. (1979). Quasi-experimentation: Design and
analysis issues for ?eld settings. Boston, MA: Houghton Mif?in
Company.
Dillman, D. (1978). Mail and telephone surveys: The total design method.
New York, NY: Wiley.
Dillman, D. (1999). Mail and internet surveys: The tailored design method.
New York, NY: Wiley.
Graham, J., Harvey, C., & Rajgopal, S. (2005). The economic implications of
corporate ?nancial reporting. Journal of Accounting and Economics,
40(1–3), 3–73.
Hausman, D. (1998). Causal asymmetries. Cambridge, UK: Cambridge
University Press.
Hesford, J., Lee, S., Van der Stede, W., & Young, S. M. (2007). Management
accounting: A bibliographic study. In C. Chapman, A. Hopwood, & M.
Shields (Eds.), Handbook of management accounting research
(pp. 3–26). Oxford, UK: Elsevier.
Innes, J., Mitchell, F., & Sinclair, D. (2000). Activity-based costing in the
U.K’.s largest companies: A comparison of 1994 and 1999 survey
results. Management Accounting Research, 11(3), 349–362.
Katz, D., Kahn, R., & Adams, J. (1980). The study of organizations. San
Francisco, CA: Jossey-Bass.
Kross, E., Verduyn, P., Demiralp, E., Park, J., Lee, D. S., Lin, N., et al. (2013).
Facebook use predicts declines in subjective well-being in young
adults. PLoS One, 8(8).http://dx.doi.org/10.1371/journal.pone.
0069841.
Libby, T., & Lindsay, M. (2010). Beyond budgeting or budgeting
reconsidered? A survey of North-American budgeting practice.
Management Accounting Research, 21(1), 56–75.
Luft, J., & Shields, M. (2014). Subjectivity in developing and validating
causal arguments in positivist accounting research. Accounting,
Organizations and Society, 39(7), 550–558.
Luft, J., & Shields, M. (2003). Mapping management accounting: Graphics
and guidelines for theory-consistent empirical research. Accounting,
Organizations and Society, 28(2–3), 169–249.
Merchant, K. (1990). The effects of ?nancial controls on data
manipulation and management myopia. Accounting, Organizations
and Society, 15(4), 297–313.
Merchant, K. (2012). Making management accounting research more
useful. Paci?c Accounting Review, 24(3), 334–356.
Mitchell, T., & James, L. (2001). Building better theory: Time and the
speci?cation of when things happen. Academy of Management Review,
26(4), 530–547.
W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574 573
Rein, M. (1976). Social science and public policy. Harmondsworth, UK:
Penguin.
Rind?eisch, A., Malter, A., Ganesan, S., & Moorman, C. (2008).
Cross-sectional vs. longitudinal survey research: Concepts,
?ndings, and guidelines. Journal of Marketing, 45(June),
261–279.
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-
experimental designs for generalized causal inference. Boston, MA:
Houghton Mif?in.
Speckbacher, G., Bischof, J., & Pfeiffer, T. (2003). A descriptive analysis of
the implementation of balanced scorecards in German-speaking
countries. Management Accounting Research, 14(4), 361–388.
Van der Stede, W., Young, S. M., & Chen, C. (2005). Assessing the quality of
evidence in empirical management accounting research: The case of
survey studies. Accounting, Organizations and Society, 30(7–8),
655–684.
Woodward, J. (2003). Making things happen: A theory of causal explanation.
Oxford, UK: Oxford University Press.
574 W.A. Van der Stede / Accounting, Organizations and Society 39 (2014) 567–574

doc_450339637.pdf
 

Attachments

Back
Top