VaRy complex | FT Alphaville

VaRy complex

Two rather disarming charts from Dresdner today.

They’re looking at the distribution of Goldman Sachs’ daily trading profit and loss numbers, to show just how different risk has become in the current environment — and conversely, how susceptible banks’ risk models are in stressed markets.

From the note by Dresdner’s Nigel Myer and Folkert Jan Van Der Veer:

One might reasonably expect, given the ubiquity with which the normal distribution is used in finance, that this frequency chart might display something approaching a normal distribution with a modal group in modestly positive territory. This is indeed what we see for most of the early part of the century with 2003 being a very good example of what theory would suggest to be the ‘right’ result.

Dresdner - 2003 GS daily trading P&L distribution

Roughly speaking, that means that most days in 2003 produced profits of between $20-$40m. Exceptionally good days, of which there were about 18, produced profits above $80m. Now compare that to this chart:

Dresdner - 2008 GS daily trading P&L distribution

That’s Goldman’s daily profit and loss distribution in the extreme markets of 2008. You can see that ‘normal distribution’ has flown the coop. Fat tails become commonplace – with 90 days of profits above $100m and 36 days of losses exceeding $100m.

Looking at the bank’s Value at Risk model (which uses a one-day 95 per cent VaR) – it’s equally extreme. VaR exceptions are reported on two days for the four-year period between 2002 and 2005. In 2006 exceptions rose to three, and in 2007 and 2008 to 10 days and 13 days respectively.

So VaR ended up being a very bad measure of tail risk. That’s not really news, but it is a big deal since the mathematical model is supposed to measure the potential losses of a portfolio — thereby helping banks and regulators manage and gauge risk. Yet clearly there’s been a lack of tail-risk awareness.

Stumbling and Mumbling has an interesting take on the issue:

I just can’t believe banks did this as an intellectual error, because it’s easy to incorporate tail risk into VAR. This is because we do have a statistical description of tail risk – the power law… Any idiot can tweak VAR models to incorporate fat tails… I just can’t believe that banks’ risk managers have been so stupid as to be unable to do this. It’s certainly not the case that fat tails are something new; we’ve known about them since at least October 1987. Instead, I suspect that banks’ failure lay elsewhere. One difficulty was that their portfolios were much more complex than mere equities, so it was harder to calculate the distribution of extreme returns…

The complexity of banks’ portfolios is echoed by Dresdner as well (emphasis ours):

The illiquidity of markets is probably the greatest challenge to risk controls and the biggest destroyer of model validity. VaR models assume the ability to liquidate any position on demand at the stated valuation. That means risk can be calculated by looking at the daily volatility of prices to create an expectation of what is likely. However, the events of last year showed that to be a flawed approach. Many assets have been stuck, unsaleable at any price; others were on trading books but in effect only marked intermittently. So, daily VaR calculations might have suggested low risk on assets which subsequently collapsed in value. Managing to an inappropriate model is very likely to have led to some very unpleasant surprises in dealing rooms the world over.

This means, to us, that one of the outcomes from the crisis needs to be better measures of risk as many of the old ‘certainties’ clearly no longer apply. We have long regarded model risk as the biggest threat to financial institutions because of its ability to deliver huge negative surprises from supposedly stable situations. Not everything is normal in finance; as The Economist recently pointed out, under the normal distribution the Dow should have moved by more than 4.5% on 6 days in the period between 1916 and 2003, when in fact it moved by this amount on 366 days.

Loss making tail events, for banks, will occasionally be down to a single, vast, isolated loss; writing down exposures to Madoff or Enron are good examples. Most of the time, large daily losses are going to be due to moderate losses in a significant number of business areas at the same time. Much of financial theory holds that this should only happen very rarely, but we suspect that this is not the case in the real world. Theory holds that most asset classes, or at least trading books, are going to be independent and hence when a loss strikes one book, there is little or no correlation into other books. But that is clearly not the case (perhaps in part because of the tendency of many risk takers to take risk, or hedge it, with products outside their direct remit) as markets and asset classes tend to have high correlations in bad times and only behave more independently in more stable times when specific information drives one asset class but not another.

With that in mind, we think that the future is going to require a much better understanding of correlations between risk classes in times of stress. Understanding alone won’t reduce risk, but it is possible that managements may use that understanding to manage risks better from a top down perspective – setting more appropriate goals and reward structures. They may also have to be more explicit in their willingness to run illiquidity risks and investors may have to change their expectations of returns as well.

So while it’s good news that banks are starting to look into the viability of their mathematical models, the picture painted by Dresdner and Stumbling and Mumbling above suggests there are deeper issues at play; underestimating tail risks has just as much to do with to do with the structure and management within banks themselves as it does to do with models — and there’s no easy fix for that.

Related links:
Lord Turner, VaR and the FSA – FT Alphaville
Don’t blame models – Stumbling and Mumbling
Quant blame me – FT Alphaville