We’ve got some big plans
is one of the scarier things for a central banker to say, but Andrew Bailey did and we’re now, like everyone, on tenterhooks for the publication of Ben Bernanke’s review into the Bank of England’s forecasting methods, which will land soon.
MainFT’s Sam Fleming spoke to the Guv’nor following the March Monetary Policy Committee meeting, with Bailey revealing that the BoE’s infamous fan charts are not long for this world:
A growing number of BoE officials are in favour of instead using projections of alternative scenarios, potentially alongside a central forecast. But analysts and officials stress that shifting to a new regime will not be easy and caution that the reforms are unlikely to be brought in rapidly.
Here’s a reminder of how we got here: the forecasts were bad.
More specifically, the Bank consistently underestimated the future level of inflation as it went up during the recent cycle, then overestimated it on the way back down.
Here, via Rabobank’s Stefan Koopman — who has written an interesting review preview — is how those errors look as a chart:
First things first — did these errors matter? It’s a complicated question that we immediately regret posing. Net satisfaction with the BoE’s inflation-management performance fell pretty much in sync with inflation rising, and we suppose that, for a variety of practical and idealistic reasons, it’s bad for people to hold their central bank in contempt. So . . . yes?
But maybe, just maybe , once the conditions were in place for an unprecedented, multipronged supply-side shock, there never was much hope.
Regardless, Bernanke was brought in last summer to review “forecasting and related processes”. The brief terms of reference can be found here . They say:
the review should consider the appropriate approach to forecasting and analysis in support of decision-making and communications in times of high uncertainty from big shocks and structural change focussing on: — staff processes and analysis supporting the MPC’s policy deliberations; — the analytical framework for taking account of significant shocks and shifts on the supply as well as the demand side of the economy; — the role of the forecast in the MPC’s policy decisions and communications, including the roles of the MPC and the staff in the development of the official forecast; — the appropriate conditioning assumptions in projections, including the interest rate path on which the forecast is based; — material provided to the MPC to assist the discussion and communication of the outlook and the risks around that.
There’s a lot to think about there. On the first point, MainFT reports Bailey said the changes coming will include “entail increased investment in the bank’s IT systems”.
We’re obviously not experts on Threadneedle Street’s computing infrastructure (or anything else), but this statement calls to mind a horrific fatberg of degraded systems, linchpin Excel spreadsheets, servers that have been running since the 1990s, and COBOL. In that uniquely IT-ish “everything is systems” kind of way, it’s probably actually the most important thing happening here.
The second point gets us more into the warm, welcoming economic weeds. The heart of the Bank of England’s forecasting model COMPASS (Central Organising Model for Projection Analysis and Scenario Simulation), a New Keynesian general equilibrium model. It works like this :
COMPASS is been at the centre of BoE forecasting since 2011 — officials say it is “smaller and simpler” than the older models. Part of this smallness and simplicity lies in its limited scope: COMPASS provides forecasts across just fifteen variables, such as GDP, inflation, interest rates, trade, wages and consumption.
As Rabo’s Koopman puts it:
The key mechanism of the forecasts is that the central bank ends up with output-gap-driven estimates of inflation, arising from real and nominal frictions to exogenous supply or demand shocks. These estimates then drive monetary policy.
The data sample period that underpins COMPASS runs from 1993 to 2007. Now, most of the best things come from the ’90s ( ed.note: 80s ) but shock-proofed foundations for inflation targeting regimes are not one of them. After all, basically nothing happened during that time.
Koopman cont. (his emphasis):
COMPASS’s out-of-sample forecasting power in the face of the UK’s three significant supply shocks (i.e., Brexit, pandemic, war) was therefore destined to be weak: the data used to estimate the model’s parameters have never seen these kinds of volatility!
Absent from COMPASS is ‘esoteric stuff’, such as financial markets, banks, shipping, energy or other systemically significant commodities, or industrial policies. But these are breeding nests of potential ‘black swans’ that produce structural breaks and create large forecast errors. To deal with this, MPC and staff use other tools and models to cross-check findings and to supplement blind spots.
This kind of uncertainty is why, in designing a monetary policy regime, you might try putting together a monetary policy committee , in which a range of different expert views can be heard and (hopefully) integrated into forecasts.
Anyway, that’s somewhat besides the point: let’s talk about the problems that appear inherent to the forecast model. Koopman identifies three:
1) The Bank’s errors in forecasting seem to be serially correlated , which is to say they are somewhat predictable, rather than random. Writes Koopman:
The presence of such serial correlation implies that something is going wrong. While we can only speculate on the precise cause, a plausible explanation is that general equilibrium models pull towards equilibrium, i.e. 2% inflation, and that the subsequent adjustments of the model output are steered by past inflation experiences.
2) The “endless loop” conundrum. The Bank’s forecasts are conditioned on market-implied expectations for interest rates, which seems sensible until you think about it. Here, roughly, is how it works in practice:
Bernanke himself once described steering monetary policy by markets movements as a “ hall of mirrors ” problem, but that was harder to pithily illustrate.
3) Communication is hard, but also too easy. Media ethics alarm !!! The Bank says lots of things and, particularly if Bailey says them, people tend to listen. Koopman:
The world’s financial press, situated at the Bank’s doorstep . . .
. . . reports on these fluctuations. This then easily permeates into the UK’s non-specialist media, which has a much broader audience. It distinguishes the UK from other European countries: changes in the ECB’s forecasts largely go unnoticed in Dutch media, for example. Consequently, expectations regarding future monetary policy decisions could more potently influence the present, affecting wages and prices possibly before ‘physical’ demand and supply conditions have changed.
It’s an interesting idea that we must admit we don’t think about much — that the UK’s high levels of media saturation mean that BoE communications are probably the subject of disproportionate scrutiny and banal coverage , which then has practical repercussions for the macroeconomic impacts of those communications. More on that shortly.
Koopman proposes two solutions:
1) The BoE begins doing scenario analysis:
When potential inflation outcomes could vary widely, for example due to overlapping supply shocks, different analytical frameworks yield different inflation forecasts. Confidence in a specific inflation path is therefore fairly low. In such cases, it is helpful to create multiple forecasts or scenarios that reflect different states of the world.
(Sorry to triple-link but this is discussed well in Sam’s piece .)
A thought. Briefly, here’s how economics journalism works in parts of Britain’s vaunted press: — The Bank of England publishes a fan chart showing different forecasts of the trajectory of the economy — Journalists focus on the scariest or least scary one (depending on narrative priors), because those make for the most engaging headlines and copy — The extreme scenarios don’t occur — Journalists dunk of the BoE for bad forecasting
Hypothetically, this situation damages trust in the BoE, which is probably bad for the reasons not specified above.
Now, a question: does a multiple forecast/scenario model rectify this problem at all?
2) The MPC does it own forecasts:
This not only rectifies the misspecification, but also improves transparency. The most efficient course of action to meet the inflation target is to unequivocally articulate the interest rate path that is required to get there.
. . . :
Because the review is being led by Ben Bernanke, who — while chairing the Federal Reserve — introduced DOT PLOTS, there’s reasonable expectation that using a DOT PLOT might be one of his suggestions. For those unfamiliar with DOT PLOTS: in such a system, each member of a rate-setting committee presents their base case for appropriate future rates. These are then plotted as dots — a DOT PLOT, if you will.
Not so fast, says Koopman:
However, the dot plot’s efficacy is debatable. It still tends to be interpreted by markets and media as a collective commitment, (e.g., “The ed says it’s going to hike three times this year”), instead of a collection of individual views. Secondly, the anonymity of the dots and their separation from economic forecasts sometimes add to the confusion, leaving the rationale behind some of the dots unclear.
Even though Bernanke introduced the dot plot at the Fed, we don’t think a dot plot would be particularly helpful for the Bank of England. The FOMC consists of seven members of the board of governors and twelve regional Federal Reserve Bank presidents, including non-voting members that want to steer the discussion. They also all have their own research staffs with varying forecasting models and perspectives. Meanwhile, the Bank of England MPC consists of nine policy makers and a common staff. This makes it much easier for the Bank of England to find common ground and to provide its reaction function, together with a collective projection of key macro variables…
He concludes:
By proactively outlining its reaction functions for various scenarios, the MPC has a better shot at shaping economic narratives instead of reacting to it. However, it may require time for the market to adapt to and to accurately interpret a new strategy.
There’s much to consider. And, judging by Bailey’s comments, this is (other than the looming death of the fan chart), potentially a “live” review — the recommendations, and how the BoE may respond to them, are an open question. Which is, you know, nice. But we’re not entirely convinced this is something that can be fixed.