Description
Forecasting is the process of making statements about events whose actual outcomes (typically) have not yet been observed. A commonplace example might be estimation of some variable of interest at some specified future date. Prediction is a similar, but more general term.
Suomen Pankki Bank of Finland P.O.Box 160, SF-00101 HELSINKI, Finland = + 358 0 1831
BANK OF FINLAND DISCUSSION PAPERS
23/96
Juhana Hukkinen Matti Vir6n Economics Department 26.9.1996
-
Assessing the Forecasting Performance of a Macroeconomic Model
ISBN 95 1-686-519-4 ISSN 0785-3572 Suomen Pankin monistuskeskus Helsinki 1996
Assessing the Forecasting Performance of a Macroeconomic Model
Bank of Finland Discussion Papers 23/96
Juhana Hukkinen
Economics Department
Matti VirCn
Research Department
Abstract
This paper contains a description of a small quarterly forecasting model for the Finnish economy. We evaluate the forecasting properties of the model by means of stochastic simulation involving both the endogenous and exogenous variables of the model. The simulations allow us to identify and quantify the main sources of forecasting uncertainty. We are also able to assess the linearity of the model. Forecasting performance is also analyzed in a conventional way by means of dynamic simulation. The important issue in these simulations is the stability of the model: how simulated values depend on the estimation period and the ordering of time periods. Key words: forecasting, macro models, simulation
Tiivistelma
Keskustelualoitteessa k&itelli%inSuomen Pankissa rakennetun neljannesvuosimallin (QMED-mallin) nykyista versiota. Tassa yhteydessa esitellaan paitsi mallin rakennetta ja mallin kayttoon liittyvia asioita myos stokastisten simulointien tuloksia. Niiilla simuloinneilla pyritsn selvittamaan mallin perusominaisuuksia ennustekaytossa. Talloin kay ilmi se, miten seka endogeenisiin etta eksogeenisiin muuttujiin liittyvat ennustamattomat satunnaistekijat ilmenevat ennusteiden epavarmuutena. Myos mallin lineaarisuutta voidaan arvioida. Ennustekykya arvioidaan tavanomaiseen tapaan dynaamisten simulointien avulla. Niiiden simulointien paatarkoituksena on mallin stabiilisuusominaisuuksien selvittaminen. Stabiilisuutta tarkastellaan tutkimalla simuloitujen arvojen riippuvuutta estimointiajanjaksoistaja havaintoajanjaksojen jarjestyksestk Asiasanat: ennustaminen, makromallit, simulointi
Contents
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 . 1 Introduction
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 .
. 2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Some computational properties of the model . . . . . . . . . . . . . . . . . . . . .7 8 2.2 Results of the stochastic simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Analysis of stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 .
3 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 .. References Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18 .
The contents of the model
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
1
Introduction
This paper reports some basic results obtained with a small Finnish quarterly model developed at the Bank of Finland, where it is used mainly for short-term forecasting. The model is called the QMED (Quarterly Model of the Economics Department - of the Bank of Finland). We focus on the simulation properties of the model. First, we scrutinize the properties of the model by means of stochastic simulation, using the procedure suggested by Brown and Mariano (1981) with the actual residuals. For the sake of comparison, several simulations are also performed using Monte Carlo -generated data. The simulations concern both the overall sensitivity of the model and the sensitivity of the model in terms of exogenous variables. The purpose of these simulations is to assess the level of forecasting uncertainty: uncertainty stemming from both endogenous and exogenous variables. This analysis boils down to the computing of certain confidence intervals for a recent model forecast. The simulations also make it possible to examine the linearity of the model. This issue is crucial, for instance, in evaluating the values of various dynamic multipliers of the model. In addition to stochastic simulation, we analyze the stability properties of the model. Thus, we estimate the model recursively from 1976 to 1995, so that four quarters of the data are always dropped from the sample starting from the first quarter of 1976 and continuing until the last quarter of 1985. Dynamic simulation paths are computed for each estimated model version and these time paths are then compared to discern how much (estimation) sample selection affects the model's forecasts.
2
2.1
Simulation results
Some computational properties of the model
The model is basically similar to earlier versions of the model (which have been used only with a mainframe computer; see Lahti and Vir6n (1989) and Lahti (1989) for details). The main difference is that the current QMED model is somewhat smaller, having 14 stochastic equations, 10 identities and SO variables (excluding disturbances). The second difference with respect to the earlier versions of the model is that the estimation period is now 1976.1- 1995.4. The exclusion of the early 1970s is motivated by the fact that the institutional framework, particularly with respect to the capital market, is now very different from what it was in that period (not to mention the 1950s and 1960s). Thirdly, but not less importantly, the structure of the model has been changed. Now, rational expectations affect both consumption demand, interest rates and wage formation via expected inflation and the rate of change of income. In addition, the (endogenous) capacity variable has been changed so that it now corresponds to the actual capacity utilization rate obtained from the Bank of Finland investment inquiry. And the model now includes an unemployment rate equation. Otherwise the model structure is fairly standard. Output, although affected by endogenous capacity, is determined by aggregate demand; wages or, more precisely, wage drift is determined by a Phillips curve; and prices are determined
according to a mark-up model. We do not discuss the details of the model. A short presentation of the equations and variables is provided in Appendix 1. Otherwise, we refer to Huklunen and Virkn (1994), which contains a more complete description of the model. The rational expectations version of the model is solved using the Fair-Taylor (1983) algorithm. Because the model is relatively small and the maximum number of leads is only four, the computational problems are generally minimal - this is also true when working with a PC (cf Sulamaa and Virtn (1989) for further details). However, this does not mean that the model simulations are similar to those obtained with standard backward-looking expectations models. The simulated values for the sample period depend on post-sample period values. Thus, if a forecast is computed for, say, the period 1996-2000 the values of the exogenous variables both for this period and for certain subsequent periods are needed (depending on the forecast horizon and on the number of periods over which the solution path is extended in type III iterations (cf Fair and Taylor (1983)). Thus, when rational expectations models are used in forecasting, one cannot simply leave the post-forecasting period values of the exogenous variables unspecified or extrapolate them mechanically. In particular, if the forecasting (dynamic simulation) period is short and these future values are merely assumed to be constant, the simulation results are markedly different from the case where the future values are based on all available information.
2.2
Results of the stochastic simulations
Procedure The stochastic simulations were done as follows. First, we ran a standard (deterministic) dynamic simulation for the forecasting period 199641- 200044. The solution, called the baseline, is used as a point of reference for subsequent simulations, Secondly, we obtained 500 shuffled residuals for the period 199641- 200044 (the current forecasting period) from the original (OLS) residuals by means of random drawings. Thirdly, we obtained 250 pseudo values for each exogenous variable using the AR(8) model (augmented with a linear time trend) residuals of the exogenous variables (estimated for the period 197641- 199544) as a set of values from which 20 values were drawn randomly. Thus, the pseudo values , , , i = 1,2,..,20, j = 1,2,...,250, for the exogenous variables were obtained as XI, + E where eiJt is the shuffled value of the residual. In the case of exogenous variables, there is no self-evident way of carrying out the stochastic simulations. Our method is similar to the analysis of Fair (1989). The AR(4) and AR(8) residuals are used simply to get some idea of the uncertainty attached to the exogenous variables. If the time path of the variable is very smooth (volatile), it is obviously much easier (more difficult) to make correct assumptions about the future values of the variable. However, one should keep in mind that this is a very crude way of estimating the uncertainty. It turns out that the time paths of the AR model forecasts for 199641-200044 do not always make sense. Thus, if the exogenous variables are replaced by univariate AR models and the whole model is solved, the new baseline differs substantially from the original baseline. Hence, we prefer to approach the experiment by treating the exogenous variables in the "old-
fashioned" way: stochastic shocks are added to these variables to get the new pseudo variables.
Presentation of results
The results are presented as follows: The stochastic simulation results using the OLS residuals from the estimated behavioural equations are presented in Table 1. In Table 1, the results correspond to the case in which all residuals (in all stochastic equations) are taken into account. The table shows how this appears in the model forecast for these endogenous variables. The results of this exercise are also illustrated in Figures 1 and 2. These figures include the actual time paths of the shocked values of GDP and GDP prices. For expositional reasons, only 250 simulations are reported here. The bold lines represent the average values of these simulated time paths. With these data, we construct the confidence intervals (at the 95 per cent level of significance) computed as the average value & 2*(standard deviation). Figures 3 and 4 represent the corresponding time-series. The stochastic simulation results for the exogenous variables are briefly reported in Table 1, which contains the annual average errors and standard deviations with respect to the GDP forecast in the case where all exogenous variables are shocked at the same time. Table 1.
Results from stochastic~simulations
Effects of OLS residuals of all endogenous variables on Gross domestic product
-0.07 1.04 0.02 0.07 -0.03 1.50 -0.16 1.43 0.01 1.82 -0.47 1.65 -0.04 2.10 -0.84 1.89 -0.20 2.26 -1.31 2.12
GDP deflator
Effects of AR(4) residuals of all exogenous variables Gross domestic product
-0.11 0.41 -0.04 0.29 0.45 0.60 -0.03 0.35 0.53 0.64 0.06 0.44 0.58 0.56 0.22 0.43 0.70 0.58 0.23 0.41
1
!
GDP deflator
1
Average error (frrst line) refers to the percentage difference between the baseline and the 250 simulations computed over 26 quarters. The displayed figures are annual sums. Standard deviation (second line) is the corresponding statistic for the sums of quarterly standard deviations with respect to the baseline. More complete results are presented in Hukkinen and VirCn (1995). Figures 3-4 show the quarterly values.
Figure 1.
Stochastic simulation results for the growth rate of the Gross Domestic Product
Figure 2.
Stochastic simulation results for the change rate of the GDP deflator
Figure 3.
Forecast for the Gross Domestic Product
118
0
I
4
8
I
I2 16
I
20
I
l4
Time; 0=95ql,4=96ql,
...
o Avg
r
Baaeline
* Avg+2*std
e Avg-s*std
I
Displayed figures are billions of GDP at constant 1990 Finnish markka prices. The "baseline" is the deterministic dynamic simulation path, "avg" is the average of 250 stochastic simulations and "avg *2*std" is the 95 per cent confidence interval. In this and the following figures, the vertical scale represents GDP volume.
Figure 4.
Forecast for the GDP prices
0
8
12
Time;0=95q1,4=96ql,
...
16
M
a4
r
Baseline
r Avg+Z8std e Avg-s*sM
Q
Avg
I
Interpretation
In commenting on the results we start with the case in which all endogenous variables are shocked at the same time. The first thing which ought to be mentioned here concerns the spread of the simulated time paths. It is no surprise that there is a lot of volatility, but the time paths cannot be characterized as explosive. The variance increases over time, which is natural, but even after 20 quarters the forecast values of GDP concentrate very much around the baseline solution. Quarterly values are clearly more volatile than annual values. In most cases the latter values are of greater interest (by contrast, no one is interested in the value of GDP in, say, 199943). Scrutinizing the annual values reveals that the average simulation error of GDP is - 0.20 per cent for the last year of the five-year forecasting period. The corresponding standard deviation is two per cent. Thus, the 95 per cent confidence interval is about nine per cent. In the case of GDP prices (implicit GDP deflator), the average error is one (strictly speaking, - 1.3 per cent) and the standard deviation two per cent, implying a confidence interval of eight per cent. The fact that the average error of GDP and GDP prices is not exactly zero may be the result of nonnormal error terms or nonlinearities. The first explanation seems more likely, ie the distribution of estimated residuals is not in all cases normal. Thus, the residuals are clearly (negatively) skewed, and they are marked by excess kurtosis. It is no surprise that using these residuals in stochastic simulation introduces some error in the levels of the endogenous variables. Regarding the nonlinearity issue, we also carried out a stochastic simulation with genuinely normally distributed random numbers (with variance equal to the variance of the OLS residuals). It turned out that the average simulation errors in the case of the random normal variates were almost equal to zero, suggesting that the model is indeed linear. This is an important bit of information for interpreting the different policy simulations (and dynamic multipliers). In the linear case, the size of the effect depends linearly on the size of the change in the respective exogenous variable(s) and hence the effects of different policy actions do not depend on the level of the policy variable but rather on the change. We have discussed mainly the behaviour of GDP. Some comments on other variables are also called for. If we first consider the nature of the forecast uncertainty, we notice that the variable with the largest average simulation error and the largest variance is business investment. Thus, the level of investment for year 2000 can be forecast only very imprecisely. One cannot exclude the possibility that investment expenditure in 2000 will be at the same level as at the beginning of 1994, nor can one exclude the possibility that it will be twice as high as in 1994. Clearly, business investment is the weak link in the model. The reason is obvious: the investment equation involves a high degree of simultaneity. Investment both depends on and directly affects GDP. In addition to investment, wages and income are variables that are difficult to forecast. Thus, with this model, the confidence interval is very wide. This is also intuitively obvious, as it is very difficult to say anything about future incomes policy, ie whether future wage settlements will be moderate or excessively high. In the past (ie in the estimation period 1976- 1993), both regimes can be detected. Finally, we turn to the results of the analysis of exogenous variables. The results in Table 1 suggest that if uncertainty with respect to the future values of exogenous variables is somehow proportional to the variance of the AR(8) residuals of the
respective variable, the resulting GDP effects are quite small. This is true both in terms of bias and variability of simulated forecasts. None of the variables is strikingly bad in this respect. Not surprisingly, export prices, volume of eastern trade, inventory investment and the interest rate differential between Finland and Germany are the variables making the largest contribution to GDP forecasts.
2.3
Analysis of stability
Next, we analyse the models performance in mimicking Finnish business cycles fluctuations. Thus, we examine the dynamic simulation paths for the period 1985- 1995. The period was an exceptionally volatile one in Finnish economic history. First the country experienced a very strong boom (which was fuelled by a very favourable terms of trade development and liberalization of financial markets). Then the collapse of Eastern trade came down hard on an over-heated, excessively indebted and poorly competitive economy, causing an exceptionally severe depression in 1990. Thus, gross domestic product decreased by about 13 per cent over the three year period 1991- 1993 while unemployment increased from less than 100 000to over 500 000 (ie to about 20 per cent in terms of the unemployment rate).' In 1985- 1995 the sum of absolute changes in the GDP growth rate was more than 40 percent age points, which illustrates the difficult task of forecasting the future cyclical developments. Here, we examine whether the QMED model can track the actual time path of GDP and GDP prices for the period 1980- 1995. The tracking exercise itself is quite conventional, but here we go beyond the usual practice of computing a single dynamic simulation path for the period of interest in that we also compute a backward dynamic simulation path (reversing the order of the time periods). More importantly, we pay considerable attention to the stability properties of the model. Thus, we compute eleven alternative dynamic simulation paths by reestimating the model from eleven consecutive time periods. The first period is 197641- 199544, the second 197741- 199544 and so forth. Thus, we have eleven different parameter vector estimates which we use to produce the alternative dynamic simulation paths. The results of this exercise are reported in Figures 5-7. In Figure 5 we have three conventional dynamic simulation paths for the standard version of the model (estimated from the data of 197641- 199544). In Figure 6 static simulation paths are presented for GDP using both forward and backward simulation (the time horizon in these simulations is of course one quarter). In the latter case the order of the time periods is reversed so that time goes from 199544 to 198041. In Figures 7 and 8, the time horizon of both (now dynamic) simulations is set at four quarters. Finally, in Figure 9 we report the differences between actual and simulation values of GDP and the GDP deflator. The differences are computed for the endpoint values of the 1985- 1995 simulations (in terms of the actual 1995 value of GDP). Clearly, the tracking performance of the model is very good; both the upturn and downturn in GDP are correctly forecast. Also the slowdown of inflation is well explained by the model. Although it is often pointed out that dynamic simulation is
-
See Bordes, Currie and Soderstrom (1993) or Dornbush, Goldfain and Valder (1995) for a more detailed description of the Finnish crisis.
'
Figure 5.
Dynamic simulation results for 1980- 1995
Figure 6.
Quarterly static simulation results for 1976- 1995
Figure 7
Four-quarter dynamic simulation results for 1976-'1995
Figure 8.
Four-quarter backward dynamic simulation results for 1995- 1976
Figure 9.
Simulation errors for the endpoint values of 1985- 1995 simulation, %
76
77
78
79
80
81
82
83
84
85
i
%
1
I I 1
i
1
i .
a
3
9
;
1
iI
P
8
I
I
r
I
1
I
I I
I
I
I
I
I
I
not a means of prooving the validity of restrictions imposed on the model (cf Pagan (1989)), the performance is so strikingly good that we consider it to provide same justification for using the model for forecasting purposes.' More importantly, the simulation results seem to be highly robust in terms of the estimation period. Thus, for instance, dropping the first ten years of the original sample period only slightly shows up in the endpoint values of GDP (the same result also holds in terms of the whole simulation path). If the model estimated from the period 1976- 1995 is used in simulation, the actual 1995 value of GDP is exceeded by 3.5 per cent. If the estimation period is limited to 1985- 1995, the corresponding number is 1.5 per cent. Given the large changes in GDP over the period 1985- 1995 (the sum of absolute changes in growth rates equalling 40 percentage points), these differences appear to be quite unimportant. In the case of the GDP deflator, the difference is larger (from +2.5 per cent to - 1.5 per cent) suggesting that the high inflation period, 1976- 1985, shows up in the estimated parameters of the model and the model is not completely immune to the regime change which took place in the mid-1980s. One should not however exaggerate the difference, which averages only 0.4 % per annum. One may compare this with the difference between the inflation rates for the first and second halves of the estimation period. For 1976- 1985 the average inflation rate was 9.1 per cent, while for 1986- 1995 the corresponding figure was only 3.5 per cent. Thus, one might admit that the model does not fully take into account the change in inflation regime in the 1980s, but the model's performance is still relatively good. Although we end up with a relatively comforting conclusion as to the QMED model's performance, it is clear that in general one should investigate carefully whether models used for macroeconomic forecasting are crucially dependent on the specific data values for the estimation period.
See eg Fisher and Wallis (1990) for an analysis of the traclung performance of UK models. See also Brinner (1988) and Brunner and Kamin (1994) for dynamic similation exercises for the US and Japanese economies.
Concluding remarks
The QMED model seems to be fairly reliable in macroeconomic forecasting. There appear to be no systematic forecast biases. Of course, there is much uncertainty regarding the forecasts, a fact which is seldom realized in working with model forecasts. In the case of the QMED model, this uncertainty seems to be related closely to the business investment equation, which obviously requires more careful analysis in the future. As far as the exogenous variables are concerned, our analysis did not detect a single most important source of uncertainty. It is however very difficult to analyze uncertainty associated with exogenous variables. The distinction between endogenous and exogenous variables is, after all, quite arbitrary and it should perhaps be reconsidered at least in a forecasting context. In this paper, the model was used as an analytical device to examine the reasons behind recent cyclical movements in the Finnish economy. In particular, the strong boom in the late 1980s and the recent recession were analyzed. The model appears to be able to track the actual time paths of output and prices reasonably well, and this performance seems to apply also to alternative versions of the model (where the versions differ in terms of estimation period for determining the parameter values). This stability property in important because it suggests that the model may not be excessively sensitive to different policy regimes and institutional changes. The results do not of course invalidate the Lucas critique, but they may indicate that at least the QMED model can be used for conventional simulation purposes without producing completely erroneous results in this respect. Macroeconomic forecasting models have recently met criticism from various directions. To a great extent, the criticism is well-founded. The models are indeed quite old-fashioned, only weakly data-consistent and rather cumbersome in practical forecasting work. Needless to say, much effort is needed to improve the reputation of these models. Surely, one of the most important tasks is show that the models can indeed be useful and reliable.
References
Bordes, C. - Cunie, D. - Soderstrom, H.T. (1993) Three Assessments of Finland's Economic Crisis and Economic Policy. Bank of Finland, Series C, No. 9. Brinner, R. (1985) The Power of Economic Theory and Macroeconomic Models during the Past Decade. DRI Memo. Presented in the Fifth World Congress of the Econometric Society, Boston. Brown, B. - Mariano, R. (1981) Residual Based Procedures for Predicting and Estimation in a Nonlinear Simultaneous System. Econometrica 52,321-343. Brunner, A. - Kamin, S. (1994) Determinants of the 1991-1993 Japanese Recessiion: Evidence from a Structural Model of the Japanese Economy. International Finance Discussion Papers, Board of Governors of the Federal Reserve System, No. 479. Dornbusch, R. - Goldfajn, I. - Valder, R. (1995) Currency Crises and Collapses Brookings papers on Economic Activity 2:95,219-293. Fair, R.C. (1989) Does Monetary Policy Matter? Narrative vs. Structural Approaches. National Bureau of Economic Research Working Paper No. 3045, Cambridge MA. Fair, C.R. - Taylor, J.B. (1983) Solution and Maximum Likelihood Estimation of Dynamic Nonlinear Rational Expectations Models. Econometrica 5 1, 1169- 1185. Fisher, P. - Wallis, K. (1990) The Historial Tracking Performance of the UK Macroeconomic Models 1975- 1985. Economic Modelling 7, 179- 197. Hatanaka, M. (1978) On the Efficient Estimation Methods for the Macro-Economic Models Nonlinear in Variables. Journal of Econometrics 8,323- 356. Hukkinen, J. - Virkn, M. (1994) Assessing the Performance of a Macroeconomic Model. Bank of Finland Discussion Papers No. 5/95. Kriimer, W. - Sonnberger, H. (1986) The Linear Regression Model under Test. PhysicaVerlag, Heidelberg. Lahti, A. (1989) Rational Expectations in a Macromodel: An Empirical Study. Bank of Finland, Series D:72, Helsinki. Lahti, A. - Virkn, M. (1989) The Finnish Rational Expectations QMED Model: Estimation, Dynamic Properties and Policy Results. In J. Gruber (editor) Economic Decision Models, Lecture Notes in Economic and Mathematical Systems No. 366, Springer-Verlag, Heidelberg, 283 - 304. Pagan, A. (1989) On the Role of Simulation in the Statistical Evaluation of Econometric Models. Journal of Econometrics 40, 125- 139. Sulamaa, P. - Virkn, M. (1989) Examining the Effects of Anticipated Policy Actions: Results with the Finnish Micro-QMED Model. Bank of Finland Discussion Papers 35/89, Helsinki.
Appendix
The contents of the rnodel
Endogenous variables
1. Exports xq, = 2.44 + . 4 5 ~ q ,+ - .41fq,-l ~ - .76(p~/pf),_~ + . 14fe, R2 = 0.97, DW = 1.80
2. Imports
mq, = .54 + .1 lmq,-I + .17mqt-, + .10cq, + .15iq + .41xqt - .46(pm -pd), + .37hst-, R2 = 0.95, DW = 1.74
3. Private consumption
cq, = .61 + . 9 0 ~ q ,+ -~ .08(yh - p ~ ) , + - .04(rb ~ -A ~ C , + ~ -).74Apct , - ~ - .23Aunt
+
.lOAph, + .07Apht-,
R2 = .99, DW = 2.28
i
4. Business investment
5. Residental investment
ih, = 5.13 + .34ih,-l + .25bbht-] + .l lbbht-2- .28(rb - A P ~ , + ~+ ) .04Aph,-, ,-~ - .12D76 R2 = 0.95, DW = 1.94
6. Households' disposable income
7. Wage rate
Aw, = 1.04Awct+ .OlApc,+, - .04(un -nun), + .59hst R2= .96, DW = 1.60
8. Negotiated wage rate
R2 = 0.38, DW = 1.93
9. Consumption prices
Apc, = . l lAw(1 + ltax), + .19Aw(l + ltax),_, + .20Aw(l + ltax),-, + .08Apm,
+ .08Apmt-,- .03(pc - px),-,
i
R2=0.60, DW = 1.66
10. Investment prices
A2pit= ,28A2pit-I + .46A2w(1+ ltax), + .07Agmt R2 = 0.45, DW = 1.64
11. Public consumption prices
12. Government bond yield
1
I
rb, = .O1 + .58rb,-l + . 0 6 A g ~ ,+ +.29rd, ~ + .06rdift-, + .2 1rf R2=0.92, DW = 1.12
!
I
I
I
13. Capacity utilization rate
hs, = .18 +.94hstdl+ .39Ay, - 1.81Alnt- .Oilq, - .Ol(pv -pq), R2 =.91, DW =2.19
14. Unemployment rate
un, = 7.90 + .97un,-, + .15run, - .lOhs,-, - .16Ay,
R~ =0.99, DW = 1.45 15. Gross domestic product (volume)
Y=CQ+GQ+XQ+IQ+IW-MQ
16. Gross domestic product (value)
17. Private demand
18. Total fixed investment
19. Current account
CA =PX*XQ -PM.MQ +TR +V3
20. Transfers and other expenditure
TR = .005YV + IE + V4
21. Interest expences
E l = -0.79 + .003RF.DEBT
22. Foreign debt
DEBT, = DEBT,-I + CA, + V3,
23. GDP deflator
24. Private demand prices
Lower case letters denote logarithmis, capital letters untransformed values. For space reasons, the t-values and other test statistics are not displayed.
Exogenous variables
bbf bbh D76 fe fq gq ig iw ltax lm In nun Building permits for firms Building permits for households Dummy for 197641 Exports to non-market economies Foreign import demand Public consumption Public investment Inventory investment plus statistical error Employees' social security expenses Total employment Working-age population Natural rate unemployment Foreign producer prices Implicit price deflator for residential investment House prices Import prices Import prices of oil products Export prices Bank of Finland base rate Interest rate differential between Finland and Germany Long-term interest rate for FRG, UK and USA Stock prices Statistical error in national accounts Statistical error in national accounts Statistical error in national accounts Statistical error in capital account
Pf
ph ph Pm P O Px rd rdif rf
SX
vl v2 v3 v4
BANK OF FINLAND DISCUSSION PAPERS
ISSN 0785-3572 Kari T. Sipila A Data Communication Network for Administrative Purposes within the EU. 1996.53 p. ISBN 95 1-686-492-9. (TK) Veikko Saarinen - Kirsti Tanila - Kimmo Virolainen Payment and Settlement Systems in Finland 1995. 1996. 60 p. ISBN 951-686-493-7. (RM) Harri Kuussaari Systemic Risk in the Finnish Payment System: an Empirical Investigation. 1996. 32 p. ISBN 951-686-494-5. (RM) Janne Lauha OTC-johdannaiset ja Suomen oikeus (OTC Derivatives and Finnish Law 1996.98 p. ISBN 951-686-495-3. (RATA) Jukka Ahonen - Ilmo Pyyhtia Suomen teollisuuden rakenne ja hairioalttius suhteessa muihin EU-maihin (The structure of Finnish industry and its sensitivity to shocks in comparison to other EU countries). 1996. 37 p. ISBN 95 1-686-496-1. (RP) Pekka Ilmakunnas - Jukka Topi Microeconomic and Macroeconomic Influences on Entry and Exit of Firms. 1996. 33 p. ISBN 951-686-497-X. (TU) Jaakko Autio Korot Suomessa 1862-1952 (Interest Rates in Finland). 1996. 55 p. ISBN 95 1-686-498-8. (TU) Vesa Vihriala Theoretical Aspects to the Finnish Credit Cycle. 1996. 63 p. ISBN 95 1-686-500-3. (TU) Vesa Vihriala Bank Capital, Capital Regulation and Lending. 1996. 52 p. ISBN 95 1-686-501- 1. (TU) Vesa Vihriala Credit Growth and Moral Hazard. An Empirical Study of the Causes of Credit Expansion by the Finnish Local Banks in 19861990. 1996.53 p. ISBN 95 1-686-502-X. (TU) Vesa Vihriala Credit Crunch or Collateral Squeeze? An Empirical Analysis of Credit Supply of the Finnish Local Banks in 1990-1992. 1996.50 p. ISBN 951-686-503-8. (TU) Peter Redward Structural Reform in New Zealand: A Review. 1996. 35 p. ISBN 95 1-686-504-6. (KT) Kaare Guttorm Andersen - Karlo Kauko A Cross-Country Study of Market-Based Housing Finance. 1996.45 p. ISBN 951-686-505-4. (RM) Kristina Rantalainen Valtion luotonotto ja velkapaperimarkkinat Italiassa (Government borrowing and the debt instrument markets in Italy). 1996.29 p. ISBN 951-686-506-2. (RM) 15/96 Tuomas Saarenheimo Monetary Policy for Smoothing Real Fluctuations? Assessing Finnish Monetary Autonomy. 1996. 36 p. ISBN 951-686-508-9. (KT) Mikko Ayub Elakerahoitus ja talouskasvu. Taloutta kohtaavien sokkien vaikutus kasvuun ja hyvinvointiin osittain rahastoivassa elakejarjestelmbsa (Pension Financing and Economic Growth. The Effect of Exogenous Shocks on Economic Growth and Utility in a Partly Funded Pension System). 1996.72 p. ISBN 951-686-509-7. (TU)
-
16/96
Veikko Saarinen Maksujarjestelmat ja -valineet Suomessa (Payment Systems and Payment Instruments in Finland). 1996. 34 p. ISBN 951-686-511-9. (RM) Antti Ripatti Stability of the Demand for M1 and Harmonized M3 in Finland. 1996. 31 p. ISBN 951-686-513-5. (TU) Juha Seppala - Petri Viertio The Structure of Interest Rates: Estimation and Interpretation. 1996.55 p. ISBN 95 1-686-514-3. (MO) Jukka Ahonen Suomen Pankki ja tiedotusvalineet - virkamiesten ja taloustoimittajien vuorovaikutus (The Bank of Finland and the Media - Interaction between Officials and Financial Journalists). 1996.51 p. ISBN 95 1-686-515-1. (TIE) Matti Ahokas - Harri Kuussaari - Kimmo Virolainen Johdannaismarkkinat. Maailmanlaajuinen keskuspankkikysely (The Derivative Markets: A Worldwide Central Bank Survey). 1996.43 p. ISBN 95 1-686-516-X. (RM) Peter Redward - Tuomas Saarenheimo From Policy Rate to Market Rates: An Empirical Analysis of Finnish Monetary Transmission. 1996. 29 p. ISBN 95 1-686-518-6. (KT) Juhana Hukkinen - Matti Viren Assessing the Forecasting Performance of a Macroenomic Model. 1996.22,p. ISBN 95 1-686-519-4. (KT)
doc_398693906.pdf
Forecasting is the process of making statements about events whose actual outcomes (typically) have not yet been observed. A commonplace example might be estimation of some variable of interest at some specified future date. Prediction is a similar, but more general term.
Suomen Pankki Bank of Finland P.O.Box 160, SF-00101 HELSINKI, Finland = + 358 0 1831
BANK OF FINLAND DISCUSSION PAPERS
23/96
Juhana Hukkinen Matti Vir6n Economics Department 26.9.1996
-
Assessing the Forecasting Performance of a Macroeconomic Model
ISBN 95 1-686-519-4 ISSN 0785-3572 Suomen Pankin monistuskeskus Helsinki 1996
Assessing the Forecasting Performance of a Macroeconomic Model
Bank of Finland Discussion Papers 23/96
Juhana Hukkinen
Economics Department
Matti VirCn
Research Department
Abstract
This paper contains a description of a small quarterly forecasting model for the Finnish economy. We evaluate the forecasting properties of the model by means of stochastic simulation involving both the endogenous and exogenous variables of the model. The simulations allow us to identify and quantify the main sources of forecasting uncertainty. We are also able to assess the linearity of the model. Forecasting performance is also analyzed in a conventional way by means of dynamic simulation. The important issue in these simulations is the stability of the model: how simulated values depend on the estimation period and the ordering of time periods. Key words: forecasting, macro models, simulation
Tiivistelma
Keskustelualoitteessa k&itelli%inSuomen Pankissa rakennetun neljannesvuosimallin (QMED-mallin) nykyista versiota. Tassa yhteydessa esitellaan paitsi mallin rakennetta ja mallin kayttoon liittyvia asioita myos stokastisten simulointien tuloksia. Niiilla simuloinneilla pyritsn selvittamaan mallin perusominaisuuksia ennustekaytossa. Talloin kay ilmi se, miten seka endogeenisiin etta eksogeenisiin muuttujiin liittyvat ennustamattomat satunnaistekijat ilmenevat ennusteiden epavarmuutena. Myos mallin lineaarisuutta voidaan arvioida. Ennustekykya arvioidaan tavanomaiseen tapaan dynaamisten simulointien avulla. Niiiden simulointien paatarkoituksena on mallin stabiilisuusominaisuuksien selvittaminen. Stabiilisuutta tarkastellaan tutkimalla simuloitujen arvojen riippuvuutta estimointiajanjaksoistaja havaintoajanjaksojen jarjestyksestk Asiasanat: ennustaminen, makromallit, simulointi
Contents
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 . 1 Introduction
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 .
. 2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Some computational properties of the model . . . . . . . . . . . . . . . . . . . . .7 8 2.2 Results of the stochastic simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Analysis of stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 .
3 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 .. References Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18 .
The contents of the model
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
1
Introduction
This paper reports some basic results obtained with a small Finnish quarterly model developed at the Bank of Finland, where it is used mainly for short-term forecasting. The model is called the QMED (Quarterly Model of the Economics Department - of the Bank of Finland). We focus on the simulation properties of the model. First, we scrutinize the properties of the model by means of stochastic simulation, using the procedure suggested by Brown and Mariano (1981) with the actual residuals. For the sake of comparison, several simulations are also performed using Monte Carlo -generated data. The simulations concern both the overall sensitivity of the model and the sensitivity of the model in terms of exogenous variables. The purpose of these simulations is to assess the level of forecasting uncertainty: uncertainty stemming from both endogenous and exogenous variables. This analysis boils down to the computing of certain confidence intervals for a recent model forecast. The simulations also make it possible to examine the linearity of the model. This issue is crucial, for instance, in evaluating the values of various dynamic multipliers of the model. In addition to stochastic simulation, we analyze the stability properties of the model. Thus, we estimate the model recursively from 1976 to 1995, so that four quarters of the data are always dropped from the sample starting from the first quarter of 1976 and continuing until the last quarter of 1985. Dynamic simulation paths are computed for each estimated model version and these time paths are then compared to discern how much (estimation) sample selection affects the model's forecasts.
2
2.1
Simulation results
Some computational properties of the model
The model is basically similar to earlier versions of the model (which have been used only with a mainframe computer; see Lahti and Vir6n (1989) and Lahti (1989) for details). The main difference is that the current QMED model is somewhat smaller, having 14 stochastic equations, 10 identities and SO variables (excluding disturbances). The second difference with respect to the earlier versions of the model is that the estimation period is now 1976.1- 1995.4. The exclusion of the early 1970s is motivated by the fact that the institutional framework, particularly with respect to the capital market, is now very different from what it was in that period (not to mention the 1950s and 1960s). Thirdly, but not less importantly, the structure of the model has been changed. Now, rational expectations affect both consumption demand, interest rates and wage formation via expected inflation and the rate of change of income. In addition, the (endogenous) capacity variable has been changed so that it now corresponds to the actual capacity utilization rate obtained from the Bank of Finland investment inquiry. And the model now includes an unemployment rate equation. Otherwise the model structure is fairly standard. Output, although affected by endogenous capacity, is determined by aggregate demand; wages or, more precisely, wage drift is determined by a Phillips curve; and prices are determined
according to a mark-up model. We do not discuss the details of the model. A short presentation of the equations and variables is provided in Appendix 1. Otherwise, we refer to Huklunen and Virkn (1994), which contains a more complete description of the model. The rational expectations version of the model is solved using the Fair-Taylor (1983) algorithm. Because the model is relatively small and the maximum number of leads is only four, the computational problems are generally minimal - this is also true when working with a PC (cf Sulamaa and Virtn (1989) for further details). However, this does not mean that the model simulations are similar to those obtained with standard backward-looking expectations models. The simulated values for the sample period depend on post-sample period values. Thus, if a forecast is computed for, say, the period 1996-2000 the values of the exogenous variables both for this period and for certain subsequent periods are needed (depending on the forecast horizon and on the number of periods over which the solution path is extended in type III iterations (cf Fair and Taylor (1983)). Thus, when rational expectations models are used in forecasting, one cannot simply leave the post-forecasting period values of the exogenous variables unspecified or extrapolate them mechanically. In particular, if the forecasting (dynamic simulation) period is short and these future values are merely assumed to be constant, the simulation results are markedly different from the case where the future values are based on all available information.
2.2
Results of the stochastic simulations
Procedure The stochastic simulations were done as follows. First, we ran a standard (deterministic) dynamic simulation for the forecasting period 199641- 200044. The solution, called the baseline, is used as a point of reference for subsequent simulations, Secondly, we obtained 500 shuffled residuals for the period 199641- 200044 (the current forecasting period) from the original (OLS) residuals by means of random drawings. Thirdly, we obtained 250 pseudo values for each exogenous variable using the AR(8) model (augmented with a linear time trend) residuals of the exogenous variables (estimated for the period 197641- 199544) as a set of values from which 20 values were drawn randomly. Thus, the pseudo values , , , i = 1,2,..,20, j = 1,2,...,250, for the exogenous variables were obtained as XI, + E where eiJt is the shuffled value of the residual. In the case of exogenous variables, there is no self-evident way of carrying out the stochastic simulations. Our method is similar to the analysis of Fair (1989). The AR(4) and AR(8) residuals are used simply to get some idea of the uncertainty attached to the exogenous variables. If the time path of the variable is very smooth (volatile), it is obviously much easier (more difficult) to make correct assumptions about the future values of the variable. However, one should keep in mind that this is a very crude way of estimating the uncertainty. It turns out that the time paths of the AR model forecasts for 199641-200044 do not always make sense. Thus, if the exogenous variables are replaced by univariate AR models and the whole model is solved, the new baseline differs substantially from the original baseline. Hence, we prefer to approach the experiment by treating the exogenous variables in the "old-
fashioned" way: stochastic shocks are added to these variables to get the new pseudo variables.
Presentation of results
The results are presented as follows: The stochastic simulation results using the OLS residuals from the estimated behavioural equations are presented in Table 1. In Table 1, the results correspond to the case in which all residuals (in all stochastic equations) are taken into account. The table shows how this appears in the model forecast for these endogenous variables. The results of this exercise are also illustrated in Figures 1 and 2. These figures include the actual time paths of the shocked values of GDP and GDP prices. For expositional reasons, only 250 simulations are reported here. The bold lines represent the average values of these simulated time paths. With these data, we construct the confidence intervals (at the 95 per cent level of significance) computed as the average value & 2*(standard deviation). Figures 3 and 4 represent the corresponding time-series. The stochastic simulation results for the exogenous variables are briefly reported in Table 1, which contains the annual average errors and standard deviations with respect to the GDP forecast in the case where all exogenous variables are shocked at the same time. Table 1.
Results from stochastic~simulations
Effects of OLS residuals of all endogenous variables on Gross domestic product
-0.07 1.04 0.02 0.07 -0.03 1.50 -0.16 1.43 0.01 1.82 -0.47 1.65 -0.04 2.10 -0.84 1.89 -0.20 2.26 -1.31 2.12
GDP deflator
Effects of AR(4) residuals of all exogenous variables Gross domestic product
-0.11 0.41 -0.04 0.29 0.45 0.60 -0.03 0.35 0.53 0.64 0.06 0.44 0.58 0.56 0.22 0.43 0.70 0.58 0.23 0.41
1
!
GDP deflator
1
Average error (frrst line) refers to the percentage difference between the baseline and the 250 simulations computed over 26 quarters. The displayed figures are annual sums. Standard deviation (second line) is the corresponding statistic for the sums of quarterly standard deviations with respect to the baseline. More complete results are presented in Hukkinen and VirCn (1995). Figures 3-4 show the quarterly values.
Figure 1.
Stochastic simulation results for the growth rate of the Gross Domestic Product
Figure 2.
Stochastic simulation results for the change rate of the GDP deflator
Figure 3.
Forecast for the Gross Domestic Product
118
0
I
4
8
I
I2 16
I
20
I
l4
Time; 0=95ql,4=96ql,
...
o Avg
r
Baaeline
* Avg+2*std
e Avg-s*std
I
Displayed figures are billions of GDP at constant 1990 Finnish markka prices. The "baseline" is the deterministic dynamic simulation path, "avg" is the average of 250 stochastic simulations and "avg *2*std" is the 95 per cent confidence interval. In this and the following figures, the vertical scale represents GDP volume.
Figure 4.
Forecast for the GDP prices
0
8
12
Time;0=95q1,4=96ql,
...
16
M
a4
r
Baseline
r Avg+Z8std e Avg-s*sM
Q
Avg
I
Interpretation
In commenting on the results we start with the case in which all endogenous variables are shocked at the same time. The first thing which ought to be mentioned here concerns the spread of the simulated time paths. It is no surprise that there is a lot of volatility, but the time paths cannot be characterized as explosive. The variance increases over time, which is natural, but even after 20 quarters the forecast values of GDP concentrate very much around the baseline solution. Quarterly values are clearly more volatile than annual values. In most cases the latter values are of greater interest (by contrast, no one is interested in the value of GDP in, say, 199943). Scrutinizing the annual values reveals that the average simulation error of GDP is - 0.20 per cent for the last year of the five-year forecasting period. The corresponding standard deviation is two per cent. Thus, the 95 per cent confidence interval is about nine per cent. In the case of GDP prices (implicit GDP deflator), the average error is one (strictly speaking, - 1.3 per cent) and the standard deviation two per cent, implying a confidence interval of eight per cent. The fact that the average error of GDP and GDP prices is not exactly zero may be the result of nonnormal error terms or nonlinearities. The first explanation seems more likely, ie the distribution of estimated residuals is not in all cases normal. Thus, the residuals are clearly (negatively) skewed, and they are marked by excess kurtosis. It is no surprise that using these residuals in stochastic simulation introduces some error in the levels of the endogenous variables. Regarding the nonlinearity issue, we also carried out a stochastic simulation with genuinely normally distributed random numbers (with variance equal to the variance of the OLS residuals). It turned out that the average simulation errors in the case of the random normal variates were almost equal to zero, suggesting that the model is indeed linear. This is an important bit of information for interpreting the different policy simulations (and dynamic multipliers). In the linear case, the size of the effect depends linearly on the size of the change in the respective exogenous variable(s) and hence the effects of different policy actions do not depend on the level of the policy variable but rather on the change. We have discussed mainly the behaviour of GDP. Some comments on other variables are also called for. If we first consider the nature of the forecast uncertainty, we notice that the variable with the largest average simulation error and the largest variance is business investment. Thus, the level of investment for year 2000 can be forecast only very imprecisely. One cannot exclude the possibility that investment expenditure in 2000 will be at the same level as at the beginning of 1994, nor can one exclude the possibility that it will be twice as high as in 1994. Clearly, business investment is the weak link in the model. The reason is obvious: the investment equation involves a high degree of simultaneity. Investment both depends on and directly affects GDP. In addition to investment, wages and income are variables that are difficult to forecast. Thus, with this model, the confidence interval is very wide. This is also intuitively obvious, as it is very difficult to say anything about future incomes policy, ie whether future wage settlements will be moderate or excessively high. In the past (ie in the estimation period 1976- 1993), both regimes can be detected. Finally, we turn to the results of the analysis of exogenous variables. The results in Table 1 suggest that if uncertainty with respect to the future values of exogenous variables is somehow proportional to the variance of the AR(8) residuals of the
respective variable, the resulting GDP effects are quite small. This is true both in terms of bias and variability of simulated forecasts. None of the variables is strikingly bad in this respect. Not surprisingly, export prices, volume of eastern trade, inventory investment and the interest rate differential between Finland and Germany are the variables making the largest contribution to GDP forecasts.
2.3
Analysis of stability
Next, we analyse the models performance in mimicking Finnish business cycles fluctuations. Thus, we examine the dynamic simulation paths for the period 1985- 1995. The period was an exceptionally volatile one in Finnish economic history. First the country experienced a very strong boom (which was fuelled by a very favourable terms of trade development and liberalization of financial markets). Then the collapse of Eastern trade came down hard on an over-heated, excessively indebted and poorly competitive economy, causing an exceptionally severe depression in 1990. Thus, gross domestic product decreased by about 13 per cent over the three year period 1991- 1993 while unemployment increased from less than 100 000to over 500 000 (ie to about 20 per cent in terms of the unemployment rate).' In 1985- 1995 the sum of absolute changes in the GDP growth rate was more than 40 percent age points, which illustrates the difficult task of forecasting the future cyclical developments. Here, we examine whether the QMED model can track the actual time path of GDP and GDP prices for the period 1980- 1995. The tracking exercise itself is quite conventional, but here we go beyond the usual practice of computing a single dynamic simulation path for the period of interest in that we also compute a backward dynamic simulation path (reversing the order of the time periods). More importantly, we pay considerable attention to the stability properties of the model. Thus, we compute eleven alternative dynamic simulation paths by reestimating the model from eleven consecutive time periods. The first period is 197641- 199544, the second 197741- 199544 and so forth. Thus, we have eleven different parameter vector estimates which we use to produce the alternative dynamic simulation paths. The results of this exercise are reported in Figures 5-7. In Figure 5 we have three conventional dynamic simulation paths for the standard version of the model (estimated from the data of 197641- 199544). In Figure 6 static simulation paths are presented for GDP using both forward and backward simulation (the time horizon in these simulations is of course one quarter). In the latter case the order of the time periods is reversed so that time goes from 199544 to 198041. In Figures 7 and 8, the time horizon of both (now dynamic) simulations is set at four quarters. Finally, in Figure 9 we report the differences between actual and simulation values of GDP and the GDP deflator. The differences are computed for the endpoint values of the 1985- 1995 simulations (in terms of the actual 1995 value of GDP). Clearly, the tracking performance of the model is very good; both the upturn and downturn in GDP are correctly forecast. Also the slowdown of inflation is well explained by the model. Although it is often pointed out that dynamic simulation is
-
See Bordes, Currie and Soderstrom (1993) or Dornbush, Goldfain and Valder (1995) for a more detailed description of the Finnish crisis.
'
Figure 5.
Dynamic simulation results for 1980- 1995
Figure 6.
Quarterly static simulation results for 1976- 1995
Figure 7
Four-quarter dynamic simulation results for 1976-'1995
Figure 8.
Four-quarter backward dynamic simulation results for 1995- 1976
Figure 9.
Simulation errors for the endpoint values of 1985- 1995 simulation, %
76
77
78
79
80
81
82
83
84
85
i
%
1
I I 1
i
1
i .
a
3
9
;
1
iI
P
8
I
I
r
I
1
I
I I
I
I
I
I
I
I
not a means of prooving the validity of restrictions imposed on the model (cf Pagan (1989)), the performance is so strikingly good that we consider it to provide same justification for using the model for forecasting purposes.' More importantly, the simulation results seem to be highly robust in terms of the estimation period. Thus, for instance, dropping the first ten years of the original sample period only slightly shows up in the endpoint values of GDP (the same result also holds in terms of the whole simulation path). If the model estimated from the period 1976- 1995 is used in simulation, the actual 1995 value of GDP is exceeded by 3.5 per cent. If the estimation period is limited to 1985- 1995, the corresponding number is 1.5 per cent. Given the large changes in GDP over the period 1985- 1995 (the sum of absolute changes in growth rates equalling 40 percentage points), these differences appear to be quite unimportant. In the case of the GDP deflator, the difference is larger (from +2.5 per cent to - 1.5 per cent) suggesting that the high inflation period, 1976- 1985, shows up in the estimated parameters of the model and the model is not completely immune to the regime change which took place in the mid-1980s. One should not however exaggerate the difference, which averages only 0.4 % per annum. One may compare this with the difference between the inflation rates for the first and second halves of the estimation period. For 1976- 1985 the average inflation rate was 9.1 per cent, while for 1986- 1995 the corresponding figure was only 3.5 per cent. Thus, one might admit that the model does not fully take into account the change in inflation regime in the 1980s, but the model's performance is still relatively good. Although we end up with a relatively comforting conclusion as to the QMED model's performance, it is clear that in general one should investigate carefully whether models used for macroeconomic forecasting are crucially dependent on the specific data values for the estimation period.
See eg Fisher and Wallis (1990) for an analysis of the traclung performance of UK models. See also Brinner (1988) and Brunner and Kamin (1994) for dynamic similation exercises for the US and Japanese economies.
Concluding remarks
The QMED model seems to be fairly reliable in macroeconomic forecasting. There appear to be no systematic forecast biases. Of course, there is much uncertainty regarding the forecasts, a fact which is seldom realized in working with model forecasts. In the case of the QMED model, this uncertainty seems to be related closely to the business investment equation, which obviously requires more careful analysis in the future. As far as the exogenous variables are concerned, our analysis did not detect a single most important source of uncertainty. It is however very difficult to analyze uncertainty associated with exogenous variables. The distinction between endogenous and exogenous variables is, after all, quite arbitrary and it should perhaps be reconsidered at least in a forecasting context. In this paper, the model was used as an analytical device to examine the reasons behind recent cyclical movements in the Finnish economy. In particular, the strong boom in the late 1980s and the recent recession were analyzed. The model appears to be able to track the actual time paths of output and prices reasonably well, and this performance seems to apply also to alternative versions of the model (where the versions differ in terms of estimation period for determining the parameter values). This stability property in important because it suggests that the model may not be excessively sensitive to different policy regimes and institutional changes. The results do not of course invalidate the Lucas critique, but they may indicate that at least the QMED model can be used for conventional simulation purposes without producing completely erroneous results in this respect. Macroeconomic forecasting models have recently met criticism from various directions. To a great extent, the criticism is well-founded. The models are indeed quite old-fashioned, only weakly data-consistent and rather cumbersome in practical forecasting work. Needless to say, much effort is needed to improve the reputation of these models. Surely, one of the most important tasks is show that the models can indeed be useful and reliable.
References
Bordes, C. - Cunie, D. - Soderstrom, H.T. (1993) Three Assessments of Finland's Economic Crisis and Economic Policy. Bank of Finland, Series C, No. 9. Brinner, R. (1985) The Power of Economic Theory and Macroeconomic Models during the Past Decade. DRI Memo. Presented in the Fifth World Congress of the Econometric Society, Boston. Brown, B. - Mariano, R. (1981) Residual Based Procedures for Predicting and Estimation in a Nonlinear Simultaneous System. Econometrica 52,321-343. Brunner, A. - Kamin, S. (1994) Determinants of the 1991-1993 Japanese Recessiion: Evidence from a Structural Model of the Japanese Economy. International Finance Discussion Papers, Board of Governors of the Federal Reserve System, No. 479. Dornbusch, R. - Goldfajn, I. - Valder, R. (1995) Currency Crises and Collapses Brookings papers on Economic Activity 2:95,219-293. Fair, R.C. (1989) Does Monetary Policy Matter? Narrative vs. Structural Approaches. National Bureau of Economic Research Working Paper No. 3045, Cambridge MA. Fair, C.R. - Taylor, J.B. (1983) Solution and Maximum Likelihood Estimation of Dynamic Nonlinear Rational Expectations Models. Econometrica 5 1, 1169- 1185. Fisher, P. - Wallis, K. (1990) The Historial Tracking Performance of the UK Macroeconomic Models 1975- 1985. Economic Modelling 7, 179- 197. Hatanaka, M. (1978) On the Efficient Estimation Methods for the Macro-Economic Models Nonlinear in Variables. Journal of Econometrics 8,323- 356. Hukkinen, J. - Virkn, M. (1994) Assessing the Performance of a Macroeconomic Model. Bank of Finland Discussion Papers No. 5/95. Kriimer, W. - Sonnberger, H. (1986) The Linear Regression Model under Test. PhysicaVerlag, Heidelberg. Lahti, A. (1989) Rational Expectations in a Macromodel: An Empirical Study. Bank of Finland, Series D:72, Helsinki. Lahti, A. - Virkn, M. (1989) The Finnish Rational Expectations QMED Model: Estimation, Dynamic Properties and Policy Results. In J. Gruber (editor) Economic Decision Models, Lecture Notes in Economic and Mathematical Systems No. 366, Springer-Verlag, Heidelberg, 283 - 304. Pagan, A. (1989) On the Role of Simulation in the Statistical Evaluation of Econometric Models. Journal of Econometrics 40, 125- 139. Sulamaa, P. - Virkn, M. (1989) Examining the Effects of Anticipated Policy Actions: Results with the Finnish Micro-QMED Model. Bank of Finland Discussion Papers 35/89, Helsinki.
Appendix
The contents of the rnodel
Endogenous variables
1. Exports xq, = 2.44 + . 4 5 ~ q ,+ - .41fq,-l ~ - .76(p~/pf),_~ + . 14fe, R2 = 0.97, DW = 1.80
2. Imports
mq, = .54 + .1 lmq,-I + .17mqt-, + .10cq, + .15iq + .41xqt - .46(pm -pd), + .37hst-, R2 = 0.95, DW = 1.74
3. Private consumption
cq, = .61 + . 9 0 ~ q ,+ -~ .08(yh - p ~ ) , + - .04(rb ~ -A ~ C , + ~ -).74Apct , - ~ - .23Aunt
+
.lOAph, + .07Apht-,
R2 = .99, DW = 2.28
i
4. Business investment
5. Residental investment
ih, = 5.13 + .34ih,-l + .25bbht-] + .l lbbht-2- .28(rb - A P ~ , + ~+ ) .04Aph,-, ,-~ - .12D76 R2 = 0.95, DW = 1.94
6. Households' disposable income
7. Wage rate
Aw, = 1.04Awct+ .OlApc,+, - .04(un -nun), + .59hst R2= .96, DW = 1.60
8. Negotiated wage rate
R2 = 0.38, DW = 1.93
9. Consumption prices
Apc, = . l lAw(1 + ltax), + .19Aw(l + ltax),_, + .20Aw(l + ltax),-, + .08Apm,
+ .08Apmt-,- .03(pc - px),-,
i
R2=0.60, DW = 1.66
10. Investment prices
A2pit= ,28A2pit-I + .46A2w(1+ ltax), + .07Agmt R2 = 0.45, DW = 1.64
11. Public consumption prices
12. Government bond yield
1
I
rb, = .O1 + .58rb,-l + . 0 6 A g ~ ,+ +.29rd, ~ + .06rdift-, + .2 1rf R2=0.92, DW = 1.12
!
I
I
I
13. Capacity utilization rate
hs, = .18 +.94hstdl+ .39Ay, - 1.81Alnt- .Oilq, - .Ol(pv -pq), R2 =.91, DW =2.19
14. Unemployment rate
un, = 7.90 + .97un,-, + .15run, - .lOhs,-, - .16Ay,
R~ =0.99, DW = 1.45 15. Gross domestic product (volume)
Y=CQ+GQ+XQ+IQ+IW-MQ
16. Gross domestic product (value)
17. Private demand
18. Total fixed investment
19. Current account
CA =PX*XQ -PM.MQ +TR +V3
20. Transfers and other expenditure
TR = .005YV + IE + V4
21. Interest expences
E l = -0.79 + .003RF.DEBT
22. Foreign debt
DEBT, = DEBT,-I + CA, + V3,
23. GDP deflator
24. Private demand prices
Lower case letters denote logarithmis, capital letters untransformed values. For space reasons, the t-values and other test statistics are not displayed.
Exogenous variables
bbf bbh D76 fe fq gq ig iw ltax lm In nun Building permits for firms Building permits for households Dummy for 197641 Exports to non-market economies Foreign import demand Public consumption Public investment Inventory investment plus statistical error Employees' social security expenses Total employment Working-age population Natural rate unemployment Foreign producer prices Implicit price deflator for residential investment House prices Import prices Import prices of oil products Export prices Bank of Finland base rate Interest rate differential between Finland and Germany Long-term interest rate for FRG, UK and USA Stock prices Statistical error in national accounts Statistical error in national accounts Statistical error in national accounts Statistical error in capital account
Pf
ph ph Pm P O Px rd rdif rf
SX
vl v2 v3 v4
BANK OF FINLAND DISCUSSION PAPERS
ISSN 0785-3572 Kari T. Sipila A Data Communication Network for Administrative Purposes within the EU. 1996.53 p. ISBN 95 1-686-492-9. (TK) Veikko Saarinen - Kirsti Tanila - Kimmo Virolainen Payment and Settlement Systems in Finland 1995. 1996. 60 p. ISBN 951-686-493-7. (RM) Harri Kuussaari Systemic Risk in the Finnish Payment System: an Empirical Investigation. 1996. 32 p. ISBN 951-686-494-5. (RM) Janne Lauha OTC-johdannaiset ja Suomen oikeus (OTC Derivatives and Finnish Law 1996.98 p. ISBN 951-686-495-3. (RATA) Jukka Ahonen - Ilmo Pyyhtia Suomen teollisuuden rakenne ja hairioalttius suhteessa muihin EU-maihin (The structure of Finnish industry and its sensitivity to shocks in comparison to other EU countries). 1996. 37 p. ISBN 95 1-686-496-1. (RP) Pekka Ilmakunnas - Jukka Topi Microeconomic and Macroeconomic Influences on Entry and Exit of Firms. 1996. 33 p. ISBN 951-686-497-X. (TU) Jaakko Autio Korot Suomessa 1862-1952 (Interest Rates in Finland). 1996. 55 p. ISBN 95 1-686-498-8. (TU) Vesa Vihriala Theoretical Aspects to the Finnish Credit Cycle. 1996. 63 p. ISBN 95 1-686-500-3. (TU) Vesa Vihriala Bank Capital, Capital Regulation and Lending. 1996. 52 p. ISBN 95 1-686-501- 1. (TU) Vesa Vihriala Credit Growth and Moral Hazard. An Empirical Study of the Causes of Credit Expansion by the Finnish Local Banks in 19861990. 1996.53 p. ISBN 95 1-686-502-X. (TU) Vesa Vihriala Credit Crunch or Collateral Squeeze? An Empirical Analysis of Credit Supply of the Finnish Local Banks in 1990-1992. 1996.50 p. ISBN 951-686-503-8. (TU) Peter Redward Structural Reform in New Zealand: A Review. 1996. 35 p. ISBN 95 1-686-504-6. (KT) Kaare Guttorm Andersen - Karlo Kauko A Cross-Country Study of Market-Based Housing Finance. 1996.45 p. ISBN 951-686-505-4. (RM) Kristina Rantalainen Valtion luotonotto ja velkapaperimarkkinat Italiassa (Government borrowing and the debt instrument markets in Italy). 1996.29 p. ISBN 951-686-506-2. (RM) 15/96 Tuomas Saarenheimo Monetary Policy for Smoothing Real Fluctuations? Assessing Finnish Monetary Autonomy. 1996. 36 p. ISBN 951-686-508-9. (KT) Mikko Ayub Elakerahoitus ja talouskasvu. Taloutta kohtaavien sokkien vaikutus kasvuun ja hyvinvointiin osittain rahastoivassa elakejarjestelmbsa (Pension Financing and Economic Growth. The Effect of Exogenous Shocks on Economic Growth and Utility in a Partly Funded Pension System). 1996.72 p. ISBN 951-686-509-7. (TU)
-
16/96
Veikko Saarinen Maksujarjestelmat ja -valineet Suomessa (Payment Systems and Payment Instruments in Finland). 1996. 34 p. ISBN 951-686-511-9. (RM) Antti Ripatti Stability of the Demand for M1 and Harmonized M3 in Finland. 1996. 31 p. ISBN 951-686-513-5. (TU) Juha Seppala - Petri Viertio The Structure of Interest Rates: Estimation and Interpretation. 1996.55 p. ISBN 95 1-686-514-3. (MO) Jukka Ahonen Suomen Pankki ja tiedotusvalineet - virkamiesten ja taloustoimittajien vuorovaikutus (The Bank of Finland and the Media - Interaction between Officials and Financial Journalists). 1996.51 p. ISBN 95 1-686-515-1. (TIE) Matti Ahokas - Harri Kuussaari - Kimmo Virolainen Johdannaismarkkinat. Maailmanlaajuinen keskuspankkikysely (The Derivative Markets: A Worldwide Central Bank Survey). 1996.43 p. ISBN 95 1-686-516-X. (RM) Peter Redward - Tuomas Saarenheimo From Policy Rate to Market Rates: An Empirical Analysis of Finnish Monetary Transmission. 1996. 29 p. ISBN 95 1-686-518-6. (KT) Juhana Hukkinen - Matti Viren Assessing the Forecasting Performance of a Macroenomic Model. 1996.22,p. ISBN 95 1-686-519-4. (KT)
doc_398693906.pdf