Description
This study discusses the challenges and opportunities of establishing causal inference in
empirical archival financial accounting research. Causal inference requires identification
of a theoretically predicted causal mechanism in a research setting optimized to avoid
endogenous causes and using a suitable statistical inference strategy. After briefly describing
potential research design strategies, I analyze the frequency of causal studies published
in leading business and economics journals. I identify causal studies by their abstract
including an explicit reference to their causal nature and find that they are significantly
more common in the areas of economics and finance compared to other business-oriented
research disciplines like accounting. Also, the extent to which research designs are optimized
for causal inference differs significantly between causal empirical archival studies
in the area of financial accounting and finance
Causal inference in empirical archival ?nancial accounting
research
q
Joachim Gassen
?
Wirtschaftswissenschaftliche Fakultät, C.A.S.E - Center for Applied Statistics and Economics, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
a b s t r a c t
This study discusses the challenges and opportunities of establishing causal inference in
empirical archival ?nancial accounting research. Causal inference requires identi?cation
of a theoretically predicted causal mechanism in a research setting optimized to avoid
endogenous causes and using a suitable statistical inference strategy. After brie?y describ-
ing potential research design strategies, I analyze the frequency of causal studies published
in leading business and economics journals. I identify causal studies by their abstract
including an explicit reference to their causal nature and ?nd that they are signi?cantly
more common in the areas of economics and ?nance compared to other business-oriented
research disciplines like accounting. Also, the extent to which research designs are opti-
mized for causal inference differs signi?cantly between causal empirical archival studies
in the area of ?nancial accounting and ?nance. I discuss potential reasons for this gap
and make some suggestions on how the demand for and supply of well-designed causal
studies in the area of empirical archival ?nancial accounting research might be increased.
Ó 2013 Elsevier Ltd. All rights reserved.
Introduction
Identifying causal relationships in archival data is cru-
cial whenever a researcher is interested in understanding
whether theoretical predictions manifest themselves in
data. Thus, positivistic empirical studies that aim beyond
description should allow the reader to conclude whether
the observed effect is likely to be caused by the mechanism
proposed by the study, or, in short: they should allow for
causal inference (Angrist & Pischke, 2010; Leamer, 1983).
Causal inference requires ruling out alternative expla-
nations. An observed correlation or signi?cant coef?cient
in a multivariate regression does not imply causality since
it can be the result of reverse causality, omitted correlated
variables or a miss-speci?ed functional form. A causal
study is designed so that the reader can be reasonably con-
?dent that the observed empirical relation is indeed
caused by the proposed mechanism. This aspect of re-
search design is also being referred to as the internal valid-
ity of a study.
Basing causal conclusions on archival data is challeng-
ing since archival data are not the result of a perfectly con-
trolled random experiment. As an example: Assume that a
researcher is interested in understanding whether manag-
ers that face an earnings-linked bonus plan tend to arti?-
cially in?ate reported earnings numbers.
1
We could try to
address this research question by comparing the accrual pat-
terns of earnings reported by managers with earnings-linked
bonus plans with accrual patterns reported by managers
without such a bonus plan. If we can assume that bonus
plans are randomly assigned to managers then such a re-
search strategy would be suitable to draw causal inferences.
0361-3682/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.aos.2013.10.004
q
I am grateful to an anonymous reviewer, Ulf Brüggemann, Chris
Chapman (editor), Rolf Uwe Fülbier, Christian Leuz, Bill Rees, Thorsten
Sellhorn and seminar participants at the Queensland University of
Technology as well as at Humboldt-Universität zu Berlin for helpful
comments and suggestions.
?
Tel.: +49 (0)30 2093 5764.
E-mail address: [email protected]
1
See Armstrong, Jagolinzer, and Larcker (2010), which investigates the
impact of equity incentives in managerial compensation on accounting
irregularities, as an example for a recent study in this ?eld addressing the
challenge of causal inference.
Accounting, Organizations and Society 39 (2014) 535–544
Contents lists available at ScienceDirect
Accounting, Organizations and Society
j our nal homepage: www. el sevi er. com/ l ocat e/ aos
Obviously, the central identifying assumption of the
above example (that bonus plans are randomly assigned)
is courageous to say the least. In reality, bonus plans and
managers are endogenously and simultaneously deter-
mined by, e.g., the recruiting and compensation commit-
tee. What this means is: Certain types of managers and
certain types of ?rms will have a tendency to agree on cer-
tain types of compensation packages: The bonus plan will
be endogenous to the problem at hand and not random.
What can a researcher do to address this challenge? Four
options seem feasible.
First, one could decide not to address the problem. In a
carefully written paper, this would mean choosing a non-
experimental descriptive research design, avoiding any
causal interpretation of the ?ndings and explicit caveats
at prominent places throughout the paper.
Second, the researcher can try to model the bonus plan
choice. If the researcher assumes (another identifying
assumption) that the bonus plan decision is based on ob-
servable variables only, then matching or regression ap-
proaches of standard micro-econometrics can be used to
address the endogenous nature of the bonus plan. Again,
the assumption that the determinants of the bonus plan
choice can be observed is questionable. As an example, it
seems reasonable that the unobservable psychological nat-
ure of a manager has a direct impact on earnings manage-
ment behavior. It also seems likely that compensation
committees cater to the psychological pro?le of a manager
when designing the compensation package.
If the researcher feels that the endogenous choice at
hand is at least partly based on unobservable variables,
the third potential strategy is to identify an instrumental
variable or a set of instrumental variables that are corre-
lated with the endogenous choice but have no direct im-
pact on the outcome variable of interest (here, the
earnings management choice). The problem that a re-
searcher faces when identifying a suitable instrument lies
with the impossibility to test for the validity of an instru-
ment. The use of an instrument must be justi?ed theoreti-
cally. In the area of social science, a tight theoretical
argument seems fairly unlikely in many cases.
Thus, a critical empirical researcher might be tempted
to resort to strategy number four: Identifying a setting
where bonus plans can be assumed to be exogenously im-
posed on ?rms. For example, it might be possible that some
legislation(s) at some point in time introduced a regulatory
ban of earnings-based bonus plans. Such a natural experi-
ment allows for research designs that help causal inference
by exogenously manipulating the treatment of interest.
Identifying such a setting requires institutional expertise
of the researcher.
Summing up, a causal research design based on archival
data requires (a) a clear understanding of the theoretical
mechanism (the cause-effect relationship) that the re-
searcher whishes to test, (b) a concept for a ?rst-best ran-
domexperiment that would allowher or him to test for the
existence of this mechanism, (c) information on why this
?rst-best experiment is not feasible, (d) a quasi-experi-
mental research setting that is feasible and deviates from
the ?rst-best experiment as little as possible and (e) tools
for statistical inference that address the unavoidable
shortcomings of the second-best research design (Angrist
& Pischke, 2008; Shadish, Cook, & Campbell, 2001).
Causal studies: A publication analysis
Time trends across areas of research
While several methodological surveys stress the rele-
vance of causal studies and voice the demand for a meth-
odological shift towards studies optimized for causal
inference (Antonakis, Bendahan, Jacquart, & Lalive, 2010;
Chenhall & Moers, 2007; Larcker & Rusticus, 2007; Larcker
& Rusticus, 2010; Lennox, Francis, & Wang, 2012; Roberts
& Whited, 2012; Tucker, 2010) until nowlittle evidence ex-
ists about the relative importance of causal studies in the
literature across time and research ?elds. I aim to ?ll this
gap by providing descriptive evidence about the share of
causal studies in leading journals in the area of Business
and Economics.
To identify causal studies I conduct a content analysis of
all abstracts of articles published over the 2000–2012 per-
iod in business and economics journals included in the cur-
rent Financial Times 45 journal list and indexed by the
Social Science Citation Index. The content analysis classi-
?es an article as causal whenever the abstract contains
the keyword strings ‘‘causal’’, ‘‘endogenous’’, ‘‘endogene-
ity’’ or ‘‘natural experiment’’.
2
Each journal for which at
least one article is classi?ed as causal over the 2000–2012
period is included in the subsequent analysis (42 journals,
see Appendix A for a list of the included journals). Publica-
tion, classi?cation and abstract data are taken from Thom-
son Reuters Web of Knowledge. The analysis includes a
total of 30,097 studies of which 906 are classi?ed as causal
(3.0%). I verify this measurement approach by re-evaluating
a sub-set of 136 studies that the mechanism identi?ed as
causal to identify the likelihood of generating false positives.
I ?nd 6 false positives, indicating that the number of false
positives is below 5%.
Nevertheless, this approach likely generates a signi?-
cant amount of false negatives (causal studies miss-classi-
?ed as non-causal). These false negatives can be because
authors do not stress that their results allow for causal
inference in the abstract or because they use a different
terminology. Whereas I address the second concern by
experimenting with the search strings that identify causal
studies, I am unable to rule out the ?rst concern without
evaluating the research design of 30,097 studies in detail.
It might also be that authors get increasingly aware about
the difference between causal and descriptive archival
studies over time and thus get more likely to explicitly
state in their abstract that their results allow for causal
inference. Summing up: My measure is only able to pick
up ‘‘explicitly causal studies’’. While the trends of my mea-
sure remain informative, the absolute percentages should
be viewed as a lower bound and thus interpreted with care.
2
As a robustness test, I modify this approach by adding additional
keywords like ‘‘exogenous’’, ‘‘counterfactual’’ and ‘‘instrumental variable’’.
Obviously this increases the amount of identi?ed studies while also
signi?cantly increasing the amount of false positive identi?cations. My
main inferences remain unchanged.
536 J. Gassen/ Accounting, Organizations and Society 39 (2014) 535–544
The resulting data is analyzed at the journal/publica-
tion-year level. Each observation gives the percentage of
studies of the respective journal and publication-year that
are classi?ed as causal (%CAUSAL). To allow an analysis of
the publication trends in research ?elds, the journals are
assigned to the sub-?elds ‘‘Accounting’’, ‘‘Economics’’, ‘‘Fi-
nance’’, ‘‘Management’’, ‘‘Marketing’’ and ‘‘Other’’ (see
Appendix A for the journal classi?cation). Fig. 1 gives a vi-
sual representation of the publication trends. Economics
and ?nance journals publish relatively more causal studies
than journals in other ?elds. In addition a positive time
trend is observable and it appears as if this positive time
trend is more pronounced for economics and ?nance
journals.
3
Table 1 provides descriptive test statistics for these
observations. Panel A reports the average values of %CAU-
SAL over time and sub-?elds. The overall positive time
trend is signi?cant at conventional levels regardless of
whether the full sample (531 publication-year observa-
tions,
4
t-stat 4.34 measured on a regression of %CAUSAL
on DYEAR with journal ?xed effects) or only the 13 yearly
averages are analyzed (t-stat 6.57). Panel B reports results
for three ordinary least square regression models with
%CAUSAL as dependent variable. Model I includes a set of
yearly ?xed effects (not reported) and main effects for each
sub-?eld besides ‘‘Accounting’’. Thus, the ?eld’s main effects
tests for signi?cant differences between the respective sub-
?eld and accounting. Results con?rm the visual impression
of Fig. 1 that economics and ?nance journals publish signif-
icantly more causal studies relative to accounting journals
while the differences between accounting, marketing and
management journals are insigni?cant.
Model II tests whether the different sub-?elds show
time trends with respect to the relative frequency of causal
studies. DYEAR captures a linear time trend and is coded to
be distributed between zero and one. The interaction be-
tween the time trend and the sub-?eld indicators tests
for signi?cant time trends for a speci?c sub-?eld. The re-
sults show that the time trend observed in Panel A is pre-
dominantly driven by publications in the area of
economics and ?nance. The coef?cient for the ‘‘other’’ cat-
egory is marginally signi?cant while the coef?cients for the
other sub-?elds are statistically insigni?cant, with the
coef?cient for accounting only marginally so (two-sided
probability 12.0%).
Model III tests whether the sub-?eld time trends docu-
mented by model II differ signi?cantly relative to account-
ing. Like model I, model III includes an unreported set of
yearly-?xed effects that capture the common time trend
across all sub-?elds. The sub-?elds’ main effects capture
the time invariant difference across sub-?elds relative to
the sub-?eld ‘‘Accounting’’. The interactions between the
sub-?eld indicator variables and the time trend variable
DYEAR test whether the incremental time trends of the
respective sub-?elds are signi?cantly different from the
incremental time trend in accounting. The result is signi?-
cant only for management indicating that the widening
gap between economics and ?nance on the one hand and
accounting on the other hand that can be observed in
Fig. 1 is not statistically signi?cant at conventional levels.
Nevertheless, the two-sided probability level for Econom-
ics (Finance) is 13.8% (26.6%), indicating that the visual
impression of Fig. 1 is not misleading. If anything, the
gap between economics and ?nance versus accounting in
terms of causal studies is widening over time.
5
0.0%
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
2009-2012 2005-2008 2000-2004
Accounting Econcomics Finance
Management Marketing Other
Fig. 1. Relative frequency of causal studies by ?eld and publication period.
3
It could be suspected that these ?ndings are driven by the share of
empirical archival studies increasing over my sample period. However,
based on the data presented in Oler, Oler, and Skousen (2010: 654) there is
no clear time trend in the share of archival accounting studies for the time
period 2000–2007, as the share ?uctuates around 65%.
4
The number of observations (531) is below 546 (=42 * 13) as some
journals do not span the entire period of 13 years.
5
The results of this section have been checked for robustness by (a)
changing the keywords used to identify causal studies, (b) extending the
analysis on all SSCI journals in the area of business and economics
(allowing only for time trend tests), and (c) by conducting a full text search
including the top journals in accounting, ?nance and economics. The main
inferences remain unchanged.
J. Gassen/ Accounting, Organizations and Society 39 (2014) 535–544 537
I conclude from this descriptive analysis that there is an
overall increase in (explicitly) causal studies published in
in?uential business and economics journals over time.
When different sub-?elds are considered it becomes obvi-
ous that economics and ?nance journals publish relatively
more causal studies than journals from other disciplines. In
addition the overall time trend is predominantly explained
by economics and ?nance as well, potentially widening the
gap between these and other ?elds.
Causal design of accounting studies
As a next step, I turn to studies that are classi?ed by the
content analysis as causal and are dealing with accounting
Table 1
Relative frequency of causal studies in in?uential business and economics journals.
Panel A: Yearly distribution of causal studies (%CAUSAL)
Year Accounting Economics Finance Management Marketing Other Total
2000 0.0% 5.8% 2.4% 2.7% 1.3% 1.9% 2.3%
2001 1.5% 4.4% 4.1% 4.0% 4.6% 0.8% 3.2%
2002 4.4% 4.5% 3.1% 1.8% 3.7% 2.1% 3.3%
2003 4.3% 3.2% 4.1% 2.2% 5.0% 1.6% 3.4%
2004 1.6% 3.7% 4.5% 1.7% 0.9% 2.4% 2.5%
2005 3.1% 7.6% 5.8% 0.7% 4.2% 2.5% 4.0%
2006 2.8% 4.5% 5.8% 4.5% 2.2% 3.0% 3.8%
2007 4.6% 6.6% 4.5% 3.3% 3.1% 2.5% 4.1%
2008 3.8% 7.9% 5.5% 2.5% 2.6% 1.7% 4.0%
2009 2.9% 6.0% 4.3% 3.8% 3.3% 2.7% 3.8%
2010 4.5% 8.0% 6.2% 2.0% 4.6% 2.1% 4.6%
2011 3.7% 6.5% 9.3% 1.4% 5.3% 3.4% 4.9%
2012 3.7% 11.2% 6.6% 0.6% 4.8% 3.1% 5.0%
Average 3.2% 6.1% 5.1% 2.4% 3.5% 2.3% 3.3%
2000–2004 2.4% 4.3% 3.6% 2.5% 3.1% 1.8% 2.9%
2005–2008 3.6% 6.6% 5.4% 2.8% 3.0% 2.4% 4.0%
2009–2012 3.7% 7.9% 6.6% 2.0% 4.5% 2.8% 4.6%
n (journal years) 73 65 52 103 65 173 531
Causal papers 75 206 187 83 110 245 906
Total papers 2382 4756 3534 4831 3099 11,495 30,097
Panel B: Tests for ?eld and time effects
Model I Model II Model III
Dependent variable %CAUSAL %CAUSAL %CAUSAL
Accounting 0.022
ÃÃÃ
[0.008]
Economics 0.029
ÃÃÃ
0.037
ÃÃÃ
0.015
[0.006] [0.008] [0.011]
Finance 0.019
ÃÃÃ
0.029
ÃÃÃ
0.007
[0.006] [0.009] [0.012]
Management À0.008 0.029
ÃÃÃ
0.007
[0.005] [0.006] [0.010]
Marketing 0.003 0.027
ÃÃÃ
0.005
[0.006] [0.008] [0.011]
Other À0.009
Ã
0.016
ÃÃÃ
À0.006
[0.005] [0.005] [0.010]
DYEAR
Ã
Accounting 0.021
[0.013]
DYEAR
Ã
Economics 0.048
ÃÃÃ
0.028
[0.013] [0.019]
DYEAR
Ã
Finance 0.043
ÃÃÃ
0.022
[0.015] [0.020]
DYEAR
Ã
Management À0.010 À0.030
Ã
[0.017] [0.017]
DYEAR
Ã
Marketing 0.017 À0.003
[0.013] [0.019]
DYEAR
Ã
Other 0.014 À0.006
[0.008] [0.016]
Yearly ?xed effects Yes No Yes
N 531 531 531
R
2
0.159 0.183 0.183
Notes: This Table reports %CAUSAL, the relative frequency of causal studies published in in?uential Business and Economics journals (see the Appendix for a
list). %CAUSAL is measured at the journal/publication-year level, yielding a total of 531 observations based on 906 causal studies identi?ed in a set of 30,097
abstracts. A study is classi?ed as causal when the abstract contains the strings ‘‘causal’’, ‘‘endogenous’’, ‘‘endogeneity’’ or ‘‘natural experiment’’. The abstract
data for the publication years 2000–2012 are collected from the Web of Knowledge database. The coef?cient results of Model I and III are relative to the
base category formed by accounting studies. DYEAR captures a linear time trend and is de?ned as the difference of the publication year and 2000, divided
by 12. Standard errors are reported in squared brackets.
ÃÃÃ
/
ÃÃ
/
Ã
reports two-sided signi?cance at the 1/5/10 %-level, respectively.
538 J. Gassen/ Accounting, Organizations and Society 39 (2014) 535–544
topics. For that analysis I extend the scope to use all studies
published in business and economics journals included in
the Social Science Citation Index.
6
After deleting survey
articles, experimental and theory papers as well as discus-
sions, I identify a total of 69 causal accounting studies. Out
of these, 42 are classi?ed as ?nancial accounting, 10 as man-
agerial, 11 as audit and 6 as tax studies. In terms of method-
ology, 63 are characterized as empirical archival and 6 as
?eld/survey. Panel A of Table 2 represents a breakdown by
area and methodology. As this study focuses on empirical
archival work, it is likely that the measurement approach
classifying the articles as causal biases towards identifying
empirical archival articles. Thus the breakdown by method-
ology should be interpreted with care. Panel B of Table 2
presents the 41 studies that are classi?ed as empirical archi-
val studies in the area of ?nancial accounting. I analyze the
research design of these studies to identify their causal ap-
proach. I categorize the approach into three groups: (a) stud-
ies that motivate additional robustness tests by causal
inference arguments, (b) studies that motivate their main
statistical inference strategy by causal arguments and (c)
studies that motivate their research setting by causal infer-
ence arguments. Out of the 41 studies, 28 are characterized
as studies that use causality arguments as justi?cation for
robustness checks (category a), 10 studies are characterized
as studies that base their main statistical inference strategy
on causality arguments (category b) and only 3 studies are
classi?ed as studies that use causal inference arguments as
a justi?cation for their research setting (category c).
In order to compare this distribution with related ?elds,
I use the same data screening approach to identify 85
causal empirical archival studies published in the ?nance
journals ‘‘Journal of Finance’’, ‘‘Journal of Financial
Economics’’ and ‘‘Review of Financial Studies’’. I then clas-
sify these studies into the three categories mentioned
above. The data presented in Panel B of Table 2 indicates
that in ?nance, it seems much more common to choose
research settings with the objective to optimize causal
inference. The differences across the two ?elds are statisti-
cally signi?cant (v
2
= 19.24, two-sided probability of error
This study discusses the challenges and opportunities of establishing causal inference in
empirical archival financial accounting research. Causal inference requires identification
of a theoretically predicted causal mechanism in a research setting optimized to avoid
endogenous causes and using a suitable statistical inference strategy. After briefly describing
potential research design strategies, I analyze the frequency of causal studies published
in leading business and economics journals. I identify causal studies by their abstract
including an explicit reference to their causal nature and find that they are significantly
more common in the areas of economics and finance compared to other business-oriented
research disciplines like accounting. Also, the extent to which research designs are optimized
for causal inference differs significantly between causal empirical archival studies
in the area of financial accounting and finance
Causal inference in empirical archival ?nancial accounting
research
q
Joachim Gassen
?
Wirtschaftswissenschaftliche Fakultät, C.A.S.E - Center for Applied Statistics and Economics, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
a b s t r a c t
This study discusses the challenges and opportunities of establishing causal inference in
empirical archival ?nancial accounting research. Causal inference requires identi?cation
of a theoretically predicted causal mechanism in a research setting optimized to avoid
endogenous causes and using a suitable statistical inference strategy. After brie?y describ-
ing potential research design strategies, I analyze the frequency of causal studies published
in leading business and economics journals. I identify causal studies by their abstract
including an explicit reference to their causal nature and ?nd that they are signi?cantly
more common in the areas of economics and ?nance compared to other business-oriented
research disciplines like accounting. Also, the extent to which research designs are opti-
mized for causal inference differs signi?cantly between causal empirical archival studies
in the area of ?nancial accounting and ?nance. I discuss potential reasons for this gap
and make some suggestions on how the demand for and supply of well-designed causal
studies in the area of empirical archival ?nancial accounting research might be increased.
Ó 2013 Elsevier Ltd. All rights reserved.
Introduction
Identifying causal relationships in archival data is cru-
cial whenever a researcher is interested in understanding
whether theoretical predictions manifest themselves in
data. Thus, positivistic empirical studies that aim beyond
description should allow the reader to conclude whether
the observed effect is likely to be caused by the mechanism
proposed by the study, or, in short: they should allow for
causal inference (Angrist & Pischke, 2010; Leamer, 1983).
Causal inference requires ruling out alternative expla-
nations. An observed correlation or signi?cant coef?cient
in a multivariate regression does not imply causality since
it can be the result of reverse causality, omitted correlated
variables or a miss-speci?ed functional form. A causal
study is designed so that the reader can be reasonably con-
?dent that the observed empirical relation is indeed
caused by the proposed mechanism. This aspect of re-
search design is also being referred to as the internal valid-
ity of a study.
Basing causal conclusions on archival data is challeng-
ing since archival data are not the result of a perfectly con-
trolled random experiment. As an example: Assume that a
researcher is interested in understanding whether manag-
ers that face an earnings-linked bonus plan tend to arti?-
cially in?ate reported earnings numbers.
1
We could try to
address this research question by comparing the accrual pat-
terns of earnings reported by managers with earnings-linked
bonus plans with accrual patterns reported by managers
without such a bonus plan. If we can assume that bonus
plans are randomly assigned to managers then such a re-
search strategy would be suitable to draw causal inferences.
0361-3682/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.aos.2013.10.004
q
I am grateful to an anonymous reviewer, Ulf Brüggemann, Chris
Chapman (editor), Rolf Uwe Fülbier, Christian Leuz, Bill Rees, Thorsten
Sellhorn and seminar participants at the Queensland University of
Technology as well as at Humboldt-Universität zu Berlin for helpful
comments and suggestions.
?
Tel.: +49 (0)30 2093 5764.
E-mail address: [email protected]
1
See Armstrong, Jagolinzer, and Larcker (2010), which investigates the
impact of equity incentives in managerial compensation on accounting
irregularities, as an example for a recent study in this ?eld addressing the
challenge of causal inference.
Accounting, Organizations and Society 39 (2014) 535–544
Contents lists available at ScienceDirect
Accounting, Organizations and Society
j our nal homepage: www. el sevi er. com/ l ocat e/ aos
Obviously, the central identifying assumption of the
above example (that bonus plans are randomly assigned)
is courageous to say the least. In reality, bonus plans and
managers are endogenously and simultaneously deter-
mined by, e.g., the recruiting and compensation commit-
tee. What this means is: Certain types of managers and
certain types of ?rms will have a tendency to agree on cer-
tain types of compensation packages: The bonus plan will
be endogenous to the problem at hand and not random.
What can a researcher do to address this challenge? Four
options seem feasible.
First, one could decide not to address the problem. In a
carefully written paper, this would mean choosing a non-
experimental descriptive research design, avoiding any
causal interpretation of the ?ndings and explicit caveats
at prominent places throughout the paper.
Second, the researcher can try to model the bonus plan
choice. If the researcher assumes (another identifying
assumption) that the bonus plan decision is based on ob-
servable variables only, then matching or regression ap-
proaches of standard micro-econometrics can be used to
address the endogenous nature of the bonus plan. Again,
the assumption that the determinants of the bonus plan
choice can be observed is questionable. As an example, it
seems reasonable that the unobservable psychological nat-
ure of a manager has a direct impact on earnings manage-
ment behavior. It also seems likely that compensation
committees cater to the psychological pro?le of a manager
when designing the compensation package.
If the researcher feels that the endogenous choice at
hand is at least partly based on unobservable variables,
the third potential strategy is to identify an instrumental
variable or a set of instrumental variables that are corre-
lated with the endogenous choice but have no direct im-
pact on the outcome variable of interest (here, the
earnings management choice). The problem that a re-
searcher faces when identifying a suitable instrument lies
with the impossibility to test for the validity of an instru-
ment. The use of an instrument must be justi?ed theoreti-
cally. In the area of social science, a tight theoretical
argument seems fairly unlikely in many cases.
Thus, a critical empirical researcher might be tempted
to resort to strategy number four: Identifying a setting
where bonus plans can be assumed to be exogenously im-
posed on ?rms. For example, it might be possible that some
legislation(s) at some point in time introduced a regulatory
ban of earnings-based bonus plans. Such a natural experi-
ment allows for research designs that help causal inference
by exogenously manipulating the treatment of interest.
Identifying such a setting requires institutional expertise
of the researcher.
Summing up, a causal research design based on archival
data requires (a) a clear understanding of the theoretical
mechanism (the cause-effect relationship) that the re-
searcher whishes to test, (b) a concept for a ?rst-best ran-
domexperiment that would allowher or him to test for the
existence of this mechanism, (c) information on why this
?rst-best experiment is not feasible, (d) a quasi-experi-
mental research setting that is feasible and deviates from
the ?rst-best experiment as little as possible and (e) tools
for statistical inference that address the unavoidable
shortcomings of the second-best research design (Angrist
& Pischke, 2008; Shadish, Cook, & Campbell, 2001).
Causal studies: A publication analysis
Time trends across areas of research
While several methodological surveys stress the rele-
vance of causal studies and voice the demand for a meth-
odological shift towards studies optimized for causal
inference (Antonakis, Bendahan, Jacquart, & Lalive, 2010;
Chenhall & Moers, 2007; Larcker & Rusticus, 2007; Larcker
& Rusticus, 2010; Lennox, Francis, & Wang, 2012; Roberts
& Whited, 2012; Tucker, 2010) until nowlittle evidence ex-
ists about the relative importance of causal studies in the
literature across time and research ?elds. I aim to ?ll this
gap by providing descriptive evidence about the share of
causal studies in leading journals in the area of Business
and Economics.
To identify causal studies I conduct a content analysis of
all abstracts of articles published over the 2000–2012 per-
iod in business and economics journals included in the cur-
rent Financial Times 45 journal list and indexed by the
Social Science Citation Index. The content analysis classi-
?es an article as causal whenever the abstract contains
the keyword strings ‘‘causal’’, ‘‘endogenous’’, ‘‘endogene-
ity’’ or ‘‘natural experiment’’.
2
Each journal for which at
least one article is classi?ed as causal over the 2000–2012
period is included in the subsequent analysis (42 journals,
see Appendix A for a list of the included journals). Publica-
tion, classi?cation and abstract data are taken from Thom-
son Reuters Web of Knowledge. The analysis includes a
total of 30,097 studies of which 906 are classi?ed as causal
(3.0%). I verify this measurement approach by re-evaluating
a sub-set of 136 studies that the mechanism identi?ed as
causal to identify the likelihood of generating false positives.
I ?nd 6 false positives, indicating that the number of false
positives is below 5%.
Nevertheless, this approach likely generates a signi?-
cant amount of false negatives (causal studies miss-classi-
?ed as non-causal). These false negatives can be because
authors do not stress that their results allow for causal
inference in the abstract or because they use a different
terminology. Whereas I address the second concern by
experimenting with the search strings that identify causal
studies, I am unable to rule out the ?rst concern without
evaluating the research design of 30,097 studies in detail.
It might also be that authors get increasingly aware about
the difference between causal and descriptive archival
studies over time and thus get more likely to explicitly
state in their abstract that their results allow for causal
inference. Summing up: My measure is only able to pick
up ‘‘explicitly causal studies’’. While the trends of my mea-
sure remain informative, the absolute percentages should
be viewed as a lower bound and thus interpreted with care.
2
As a robustness test, I modify this approach by adding additional
keywords like ‘‘exogenous’’, ‘‘counterfactual’’ and ‘‘instrumental variable’’.
Obviously this increases the amount of identi?ed studies while also
signi?cantly increasing the amount of false positive identi?cations. My
main inferences remain unchanged.
536 J. Gassen/ Accounting, Organizations and Society 39 (2014) 535–544
The resulting data is analyzed at the journal/publica-
tion-year level. Each observation gives the percentage of
studies of the respective journal and publication-year that
are classi?ed as causal (%CAUSAL). To allow an analysis of
the publication trends in research ?elds, the journals are
assigned to the sub-?elds ‘‘Accounting’’, ‘‘Economics’’, ‘‘Fi-
nance’’, ‘‘Management’’, ‘‘Marketing’’ and ‘‘Other’’ (see
Appendix A for the journal classi?cation). Fig. 1 gives a vi-
sual representation of the publication trends. Economics
and ?nance journals publish relatively more causal studies
than journals in other ?elds. In addition a positive time
trend is observable and it appears as if this positive time
trend is more pronounced for economics and ?nance
journals.
3
Table 1 provides descriptive test statistics for these
observations. Panel A reports the average values of %CAU-
SAL over time and sub-?elds. The overall positive time
trend is signi?cant at conventional levels regardless of
whether the full sample (531 publication-year observa-
tions,
4
t-stat 4.34 measured on a regression of %CAUSAL
on DYEAR with journal ?xed effects) or only the 13 yearly
averages are analyzed (t-stat 6.57). Panel B reports results
for three ordinary least square regression models with
%CAUSAL as dependent variable. Model I includes a set of
yearly ?xed effects (not reported) and main effects for each
sub-?eld besides ‘‘Accounting’’. Thus, the ?eld’s main effects
tests for signi?cant differences between the respective sub-
?eld and accounting. Results con?rm the visual impression
of Fig. 1 that economics and ?nance journals publish signif-
icantly more causal studies relative to accounting journals
while the differences between accounting, marketing and
management journals are insigni?cant.
Model II tests whether the different sub-?elds show
time trends with respect to the relative frequency of causal
studies. DYEAR captures a linear time trend and is coded to
be distributed between zero and one. The interaction be-
tween the time trend and the sub-?eld indicators tests
for signi?cant time trends for a speci?c sub-?eld. The re-
sults show that the time trend observed in Panel A is pre-
dominantly driven by publications in the area of
economics and ?nance. The coef?cient for the ‘‘other’’ cat-
egory is marginally signi?cant while the coef?cients for the
other sub-?elds are statistically insigni?cant, with the
coef?cient for accounting only marginally so (two-sided
probability 12.0%).
Model III tests whether the sub-?eld time trends docu-
mented by model II differ signi?cantly relative to account-
ing. Like model I, model III includes an unreported set of
yearly-?xed effects that capture the common time trend
across all sub-?elds. The sub-?elds’ main effects capture
the time invariant difference across sub-?elds relative to
the sub-?eld ‘‘Accounting’’. The interactions between the
sub-?eld indicator variables and the time trend variable
DYEAR test whether the incremental time trends of the
respective sub-?elds are signi?cantly different from the
incremental time trend in accounting. The result is signi?-
cant only for management indicating that the widening
gap between economics and ?nance on the one hand and
accounting on the other hand that can be observed in
Fig. 1 is not statistically signi?cant at conventional levels.
Nevertheless, the two-sided probability level for Econom-
ics (Finance) is 13.8% (26.6%), indicating that the visual
impression of Fig. 1 is not misleading. If anything, the
gap between economics and ?nance versus accounting in
terms of causal studies is widening over time.
5
0.0%
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
2009-2012 2005-2008 2000-2004
Accounting Econcomics Finance
Management Marketing Other
Fig. 1. Relative frequency of causal studies by ?eld and publication period.
3
It could be suspected that these ?ndings are driven by the share of
empirical archival studies increasing over my sample period. However,
based on the data presented in Oler, Oler, and Skousen (2010: 654) there is
no clear time trend in the share of archival accounting studies for the time
period 2000–2007, as the share ?uctuates around 65%.
4
The number of observations (531) is below 546 (=42 * 13) as some
journals do not span the entire period of 13 years.
5
The results of this section have been checked for robustness by (a)
changing the keywords used to identify causal studies, (b) extending the
analysis on all SSCI journals in the area of business and economics
(allowing only for time trend tests), and (c) by conducting a full text search
including the top journals in accounting, ?nance and economics. The main
inferences remain unchanged.
J. Gassen/ Accounting, Organizations and Society 39 (2014) 535–544 537
I conclude from this descriptive analysis that there is an
overall increase in (explicitly) causal studies published in
in?uential business and economics journals over time.
When different sub-?elds are considered it becomes obvi-
ous that economics and ?nance journals publish relatively
more causal studies than journals from other disciplines. In
addition the overall time trend is predominantly explained
by economics and ?nance as well, potentially widening the
gap between these and other ?elds.
Causal design of accounting studies
As a next step, I turn to studies that are classi?ed by the
content analysis as causal and are dealing with accounting
Table 1
Relative frequency of causal studies in in?uential business and economics journals.
Panel A: Yearly distribution of causal studies (%CAUSAL)
Year Accounting Economics Finance Management Marketing Other Total
2000 0.0% 5.8% 2.4% 2.7% 1.3% 1.9% 2.3%
2001 1.5% 4.4% 4.1% 4.0% 4.6% 0.8% 3.2%
2002 4.4% 4.5% 3.1% 1.8% 3.7% 2.1% 3.3%
2003 4.3% 3.2% 4.1% 2.2% 5.0% 1.6% 3.4%
2004 1.6% 3.7% 4.5% 1.7% 0.9% 2.4% 2.5%
2005 3.1% 7.6% 5.8% 0.7% 4.2% 2.5% 4.0%
2006 2.8% 4.5% 5.8% 4.5% 2.2% 3.0% 3.8%
2007 4.6% 6.6% 4.5% 3.3% 3.1% 2.5% 4.1%
2008 3.8% 7.9% 5.5% 2.5% 2.6% 1.7% 4.0%
2009 2.9% 6.0% 4.3% 3.8% 3.3% 2.7% 3.8%
2010 4.5% 8.0% 6.2% 2.0% 4.6% 2.1% 4.6%
2011 3.7% 6.5% 9.3% 1.4% 5.3% 3.4% 4.9%
2012 3.7% 11.2% 6.6% 0.6% 4.8% 3.1% 5.0%
Average 3.2% 6.1% 5.1% 2.4% 3.5% 2.3% 3.3%
2000–2004 2.4% 4.3% 3.6% 2.5% 3.1% 1.8% 2.9%
2005–2008 3.6% 6.6% 5.4% 2.8% 3.0% 2.4% 4.0%
2009–2012 3.7% 7.9% 6.6% 2.0% 4.5% 2.8% 4.6%
n (journal years) 73 65 52 103 65 173 531
Causal papers 75 206 187 83 110 245 906
Total papers 2382 4756 3534 4831 3099 11,495 30,097
Panel B: Tests for ?eld and time effects
Model I Model II Model III
Dependent variable %CAUSAL %CAUSAL %CAUSAL
Accounting 0.022
ÃÃÃ
[0.008]
Economics 0.029
ÃÃÃ
0.037
ÃÃÃ
0.015
[0.006] [0.008] [0.011]
Finance 0.019
ÃÃÃ
0.029
ÃÃÃ
0.007
[0.006] [0.009] [0.012]
Management À0.008 0.029
ÃÃÃ
0.007
[0.005] [0.006] [0.010]
Marketing 0.003 0.027
ÃÃÃ
0.005
[0.006] [0.008] [0.011]
Other À0.009
Ã
0.016
ÃÃÃ
À0.006
[0.005] [0.005] [0.010]
DYEAR
Ã
Accounting 0.021
[0.013]
DYEAR
Ã
Economics 0.048
ÃÃÃ
0.028
[0.013] [0.019]
DYEAR
Ã
Finance 0.043
ÃÃÃ
0.022
[0.015] [0.020]
DYEAR
Ã
Management À0.010 À0.030
Ã
[0.017] [0.017]
DYEAR
Ã
Marketing 0.017 À0.003
[0.013] [0.019]
DYEAR
Ã
Other 0.014 À0.006
[0.008] [0.016]
Yearly ?xed effects Yes No Yes
N 531 531 531
R
2
0.159 0.183 0.183
Notes: This Table reports %CAUSAL, the relative frequency of causal studies published in in?uential Business and Economics journals (see the Appendix for a
list). %CAUSAL is measured at the journal/publication-year level, yielding a total of 531 observations based on 906 causal studies identi?ed in a set of 30,097
abstracts. A study is classi?ed as causal when the abstract contains the strings ‘‘causal’’, ‘‘endogenous’’, ‘‘endogeneity’’ or ‘‘natural experiment’’. The abstract
data for the publication years 2000–2012 are collected from the Web of Knowledge database. The coef?cient results of Model I and III are relative to the
base category formed by accounting studies. DYEAR captures a linear time trend and is de?ned as the difference of the publication year and 2000, divided
by 12. Standard errors are reported in squared brackets.
ÃÃÃ
/
ÃÃ
/
Ã
reports two-sided signi?cance at the 1/5/10 %-level, respectively.
538 J. Gassen/ Accounting, Organizations and Society 39 (2014) 535–544
topics. For that analysis I extend the scope to use all studies
published in business and economics journals included in
the Social Science Citation Index.
6
After deleting survey
articles, experimental and theory papers as well as discus-
sions, I identify a total of 69 causal accounting studies. Out
of these, 42 are classi?ed as ?nancial accounting, 10 as man-
agerial, 11 as audit and 6 as tax studies. In terms of method-
ology, 63 are characterized as empirical archival and 6 as
?eld/survey. Panel A of Table 2 represents a breakdown by
area and methodology. As this study focuses on empirical
archival work, it is likely that the measurement approach
classifying the articles as causal biases towards identifying
empirical archival articles. Thus the breakdown by method-
ology should be interpreted with care. Panel B of Table 2
presents the 41 studies that are classi?ed as empirical archi-
val studies in the area of ?nancial accounting. I analyze the
research design of these studies to identify their causal ap-
proach. I categorize the approach into three groups: (a) stud-
ies that motivate additional robustness tests by causal
inference arguments, (b) studies that motivate their main
statistical inference strategy by causal arguments and (c)
studies that motivate their research setting by causal infer-
ence arguments. Out of the 41 studies, 28 are characterized
as studies that use causality arguments as justi?cation for
robustness checks (category a), 10 studies are characterized
as studies that base their main statistical inference strategy
on causality arguments (category b) and only 3 studies are
classi?ed as studies that use causal inference arguments as
a justi?cation for their research setting (category c).
In order to compare this distribution with related ?elds,
I use the same data screening approach to identify 85
causal empirical archival studies published in the ?nance
journals ‘‘Journal of Finance’’, ‘‘Journal of Financial
Economics’’ and ‘‘Review of Financial Studies’’. I then clas-
sify these studies into the three categories mentioned
above. The data presented in Panel B of Table 2 indicates
that in ?nance, it seems much more common to choose
research settings with the objective to optimize causal
inference. The differences across the two ?elds are statisti-
cally signi?cant (v
2
= 19.24, two-sided probability of error