Seasonality bias in the ISM… fixed? | FT Alphaville

Seasonality bias in the ISM… fixed?

Wednesday’s ISM reading in the US was solid, but maybe the best reason to look favourably on it is that you can now trust the report more than you could in recent months.

We’ve previously written about how the severity of the downturn at the end of 2008 might have borked the statistical techniques used to seasonally adjust economic data, basing this mostly on some recent analysis done by the US economics teams at Nomura and Goldman Sachs.

The Institute for Supply Management updates its seasonal adjustment factors once a year. As with almost every major economic indicator in the US, the seasonal adjustments are made by applying the Commerce Department’s X12 algorithmic program.

This year’s ISM revisions were made on Tuesday, and for the first time the problem that arose from the dramatic contraction in late 2008 is addressed explicitly:

In response to concerns that the unusually large declines in autumn 2008 associated with the recent recession that may not have been adequately handled with default settings, this year the Department of Commerce used lower thresholds (critical values) for detecting outliers.

As a result of moving averages, these changes in outlier detection affected seasonal factors both before and after 2008; therefore, ISM is making revisions to seasonally adjusted data for the past seven years rather than the customary four-year period.

Nomura and Goldman have each taken a look at the new revisions to prior months and compared them against the their own attempts to correct for seasonality biases (more detail on these in our earlier posts).

Nomura has concluded that the new seasonal factors and revisions have successfully fixed some of the biases, but the flaws haven’t been removed completely. Goldman is a bit more sanguine and notes that the revisions align very closely with its own alternative series for the ISM — and at this point anticipates that uncaptured seasonal trends won’t be much of a problem.

And both interpreted the revisions as showing mostly what they had already suspected: things weren’t as bad as we feared last summer, and correspondingly they weren’t improving as quickly as we’d hoped in the early winter.

But if you want to geek out, we’ve posted extended excerpts from the latest Nomura and Goldman reports below.

Later we’ll be back with another post on how seasonality issues might be affecting the BLS employment situation report. We also had a long chat about the issue with Nomura chief US economist Lewis Alexander and will share some additional thoughts then.

For now we’ll stick to the ISM, beginning with an excerpt from the Nomura note:

In many ways, the approach that the ISM, working with the Office of the Chief Economist of the Department of Commerce, implemented is similar to our alternative method of adjustment. As a result, the revised index is more aligned with our adjusted series than the originally-reported ISM manufacturing index. Importantly, month-to-month variation in 2011 is less pronounced in the revised series (Figure 2).

The revised ISM manufacturing index is both smoother and closer to our alternative seasonally adjusted series. For example, our alternative series suggested that the ISM manufacturing index (“PMI”) was understated by 1.9 points in August in the original release, suggesting that the actual index was 52.5 (instead of the reported 50.6). Now, the index was revised up to precisely that number, 52.5

In our earlier analysis, we found the ISM seasonal factors for both manufacturing and non-manufacturing anticipated weaker data in the fourth and first quarters, and stronger data in the second and third, relative to both the pre-financial-crisis period and our alternative estimate. Figure 3 shows that the revisions to the ISM data resolve some of these issues by showing more strength in Q2 and Q3 than the understated first report, and conversely removing part of the boost to Q1 and Q4. In retrospect, this suggests that in 2011, the second year of economic recovery, manufacturing activity likely occurred in less “fits and starts” than initial estimates suggested.

Despite the reduction of detected seasonal bias in the revised ISM series, we do not believe that it has been completely eliminated. Figure 2 continues to show some residual bias related to the sharp contraction in late 2008 and early 2009. While we do not have any illusions that our own seasonal adjustment approach, which excludes the critical months of the financial crisis for the purposes of seasonal adjustment, is ideal, we find our analysis helpful in highlighting risks around month-to-month changes and monthly forecasts. …

The biggest risk to underlying activity being misinterpreted arises from the month-to-month swings in the seasonal bias. For instance, this occurs when the bias leads to official reports overstating an indicator in one month followed by understating in the next. For both the ISM manufacturing index and Chicago PMI, this phenomenon is not pronounced in January (Figure 5). So, we cannot use the seasonal argument to explain the disappointing Chicago PMI in January 2012, which declined 2 points from December to 60.2 and was lower than expected (Bloomberg consensus forecast: 63.0).

Looking ahead, for both the ISM manufacturing index and the Chicago PMI, the most pronounced swing in the bias – when it changes from overstated to understated – occurs in the reports for February (released in March) and April (released in May) (Figure 5). This suggests that, in addition to the potential for downside risks to emerge from abroad in the coming months, the lingering effects of the financial crisis in seasonal adjustments, too, are likely to be a headwind.

And here’s Goldman:

We raised concerns about the potential for seasonal distortions to a variety of economic data in recent comments and specifically cited the ISM manufacturing index as a case study. The revised ISM series is very close to the “adjusted” series we portrayed in the latter publication (see exhibit below), and effectively reduces the amplitude of seasonal adjustments to the headline series by 3-4 points.

In other words, seasonal factors boost the raw data in the weakest winter months about 1.5 points fewer than before and subtract about 2 points fewer in August (typically the strongest month for the raw data). Technically, the revisions are actually to the adjustments for the specific component series, and in some cases these are extremely large. The most important illustration of this is the new orders index, where the “swing” in seasonal factors has shrunk by more than ten points after revision. (For example, the May 2010 adjustment previously subtracted eight points from the NSA new orders figure, while the February 2011 adjustment was +2 points. Now, these adjustments are almost identical, -2.9 points and -3.3 points respectively.) This substantially smoothes the volatility in the new orders index over the past couple of years.

Related links:
‘Tis (still) the seasonality, ISM edition – FT Alphaville
‘Tis the seasonality, hold the jolly – FT Alphaville
Still cautious heading into 2012 – Tim Duy