Something I frequently hear from those who seek to denigrate evidence-based investing is that “evidence” can be used to prove almost anything. They are, of course, perfectly correct; it can.
Last year, for example, a journalist named John Bohannon and a couple of German TV producers set out to demonstrate how easy it is to turn bad science into the big headlines behind diet fads. Their challenge was to persuade as many media outlets as possible to publicise the results of “research” which “proved” that eating chocolate every day helps you lose weight.
So they spent a few thousand euros recruiting research subjects and a doctor to run the study. They intentionally used a paltry data set of 15 participants and just three weeks of data. Over a beer-fuelled weekend with a statistician friend, they managed to torture the data sufficiently heavily to extract a technically accurate if essentially meaningless conclusion that chocolate consumption had indeed contributed to weight loss.
They submitted their paper, Chocolate with high cocoa content as a weight-loss accelerator, to 20 different journals and a few fake ones too. Within 24 hours the story was accepted for publication by dozens of publications around the world. Among the victims of the sting were Bild, Europe’s largest daily newspaper, the Daily Mail, Daily Express and Daily Star in the UK, the Irish Examiner, the Times of India, the Huffington Post and Cosmopolitan. The study also made television news in the US and Australia.
Financial news-desks are bombarded with “research” all the time, and very often it’s about as scientific as John Bohannon’s study on chocolate and weight loss. Almost invariably it has been produced or commissioned by a company with a commercial conflict of interest.
If, then, a firm that either provides or sells actively managed funds comes up with “research” that appears to endorse the use of particular active funds, or active funds in general, journalists should be naturally suspicious. Unfortunately, for whatever reason, much of this “evidence” does end up in print.
But the problem with bogus evidence in the investment industry runs far deeper. That’s because the research that fund managers use to base their decisions on which stocks to buy and sell is, to a large extent, itself conflicted. Companies struggling for attention have been known to pay for the production of what looks like research by their broker. Those brokers often take payment for their work in warrants, or the right to buy the stock they have promoted at a future date at a substantial discount.
For fund managers, like journalists, it can be very hard to work out which research is properly independent and which isn’t. It’s often only at the end of a report that the truth is revealed, with a disclaimer along the lines of, “This material is not investment research”.
The answer, then, whether you’re a journalist, an investor or a financial professional, is to be extremely wary of “research”. You need to set very high standards for any evidence you use to base decisions on. What should those standards be? I’ll explain in the third and final part of this series.
Related post:
8 things the diet and investment industries have in common
Robin is a journalist and campaigner for positive change in global investing. He runs Regis Media, a niche provider of content marketing for financial advice firms with an evidence-based investment philosophy. He also works as a consultant to other disruptive firms in the investing sector.
