Still blaming analysts for being inaccurate, slow or incapable or predicting anything remotely of use? Pah, so 2008.
Blame the data instead.
The American Finance Association has a very interesting paper on the subject, entitled “Rewriting History,” examining the Institutional Brokers’ Estimate System (I/B/E/S), which compiles analyst ratings and estimates for US companies. The significance of the data is summarised in the paper (emphasis ours):
Data are the bedrock of empirical research in finance. When there are questions about the accuracy or completeness of a data source, researchers routinely go to great lengths to investigate measurement error, selection bias, or reliability. But what if the very contents of a historical database were to change, in error, over time? Such changes to the historical record would have important implications for empirical research. They could undermine the principle of replicability, which in the absence of controlled experiments is the foundation of empirical research in finance. They could result in over- or underestimates of the magnitudes of empirical effects, leading researchers down blind alleys. And to the extent that financial-market participants use academic research for trading purposes, they could lead to resource misallocation.
And lo and behold, the authors’ look at I/B/E/S downloads (roughly one per year, between 2000 and 2007) finds that between 1.6 per cent and 21.7 per cent of observations were different from one snapshot to the next. For instance, of the 332,145 observations made by the authors on the 2003 tape, 57,770 (17.4 per cent) are changed on the 2004 tape. Changes include alterations of recommendations, additions and deletions of records and — shock — removal of analyst names entirely.
And the data anomalies aren’t exactly random.
* Relative to the 2007 tape, recommendations affected by the changes on the 2000, 2001, and 2002 tapes are too optimistic, while those on the 2003, 2004, and 2005 tapes are too pessimistic.
* Patterns in affected recommendations: The changes cluster according to three widely used conditioning variables: The analyst’s reputation, the brokerage firm’s size and status, and the boldness of the recommendation. “All-star” analysts and brokerage firms sanctioned under the Global Settlement are overrepresented among affected recommendations on the 2000 and 2001 tapes and underrepresented on later tapes. “Bold” recommendations (those far from consensus) are overrepresented among affected recommendations on all tapes.
Analyst upgrades and downgrades form the basis for a lot of empirical research on their accuracy — not to mention as inputs for figuring out the reputation (read: ability) of individual stock pickers — the “all stars” mentioned above. The very idea that two clients interested in the same time period, who obtained the data on different dates, would be analysing two very different sets of data is disconcerting, to say the least.
The good news, however, is that the discrepencies have since been ‘fixed’. Turns out, a lot of them had to do with an error in the I/B/E/S system that existed until its owner, Thomson Reuters, fixed it in spring 2007 in response to the authors’ enquiries. But, the company continues to include alterations made at the request of brokers themselves for retrospective changes to their recommendation scales (for instance, changing from a five-point scale to a three-point – buy, hold or sell), which can still result in discrepencies, according to the authors. All most confusing.
Related links:
Montier: ‘Analysts are rubbish’ – FT Alphaville
Rewriting History – American Finance Association
Buy? Sell? Hold? Delete! – FT Alphaville