Index Fund Educator Home Page

Survivorship Bias
and
Other Performance Distortions

The Evidence against actively managed mutual funds is overwhelming, but it still doesn’t tell the whole story of the failure of actively managed mutual funds to justify their extraordinary costs. What if the nature of evidence collection itself was biased against Index Funds and in favor of the traditional actively managed funds? What if the poor performance statistics for the truly bad performing traditional managed funds was simply excluded from the analysis? What if, bad as they are, actively managed funds look better than they really are? This would be bad news for active managed funds indeed. And, in fact, that’s exactly the reality we will discuss below.

Before we do however, let’s review what we already know about the comparative performance of traditional mutual funds compared to index funds. We already know from the studies reviewed under Evidence elsewhere on this web site, that index funds have outpaced the average mutual fund time and time again. We also already know from the studies reviewed under Fallacy of Persistency that there is little reason to believe that those few mutual funds that have outperformed the comparable index fund in any particular period are likely to do so in some subsequent period. But, the studies on “survivorship bias” which we review below go one step further. They demonstrate that the average mutual fund has underperformed their index fund counterpart by a significantly greater amount than the comparative performance studies detailed under Evidence suggest.

A Survivorship Bias Analogy

It’s as if one class of runners in a race (as a group) was being compared to another class of runners (say Kenyan runners versus runners from the imaginary “Slowville”). Only in this particular performance comparison, the statistician decided to exclude the runners from the Slowville group who were so slow they gave up and didn’t bother finishing the race. In this analogy, the fast “Kenyan” runners are the cost efficient and thus (for purposes of this analogy) more nimble “Index Fund” racers. The “actively managed mutual funds” are the slower (cost-burdened) Slowville runners. And those “actively managed mutual funds” who did so poorly they went out of existence were the Slowville runners who did not even finish the race. What if the statistician was not so kindly towards the slower Slowville team and counted not only the Slowville runners who finished the race, but those that dropped out as well. In such case, the quoted performance comparisons (say average distance per minute) for the slow “actively managed mutual fund” Slowville runners, as a group, would turn out to be much slower in comparison to the fast Kenyan “Index Fund” runners, as a group, than originally determined. The performance of the group of “actively managed mutual funds” would be further dragged down by including the performance of the dropouts.

This is, in fact, a significant factor in studies that have compared the performance of Index Funds with those of actively managed funds. The real losers, the ones whose performance was so bad they went out of existence (i.e., stopped running the race), were discarded, making the poor performance of actively managed funds (as a group) look better than, in fact, they actually were (in comparison to the performance of unmanaged “Index Funds”). And remember, even with this distortion in their favor, actively managed mutual funds (as a group) still significantly underperformed Index Funds! As suggested by the title of this web page, we refer to this significant measurement distortion (of leaving out the ones who didn’t finish the race) “Survivorship Bias”.

Survivorship Bias Details

Larry Swedroe, in his above referenced book, Rational Investing in Irrational Times (2002), discusses this phenomenon of Survivorship Bias as follows:

“In the most comprehensive study ever done on mutual funds, covering the period 1962-93, Mark Carhart found that by 1993, fully one third of all funds in his sample had disappeared(1).” (RIIIT, page 74)

1. Mark M. Carhart, “On Persistence in Mutual Fund Performance,” Journal of Finance (March 1997).

Mr. Swedroe, then continues as follows:

“In 1986 the then-existing 568 stock funds returned 13.4%. By 1996, the 1986 performance had magically improved to 14.7%. The 1.3% improvement was a result of the disappearance of twenty-four percent of the original funds, and the fact that the 1986 performance of the funds still in existence ten years later was used in the new computation.(2)” (RIIIT, page 74-75)

2. Wall Street Journal, April 4, 1997.

Finally, to make the issue more concrete, Mr. Swedroe, relates a specific example of the mechanics of survivorship bias and its distorting effects on reported “track record” performance:

“The story of Liberty Financial Companies is a good illustration of the potential impact on reported returns when funds are merged out of existence. Liberty had been experiencing a serious drain on it assets under management. In 1999, net outflows were over $600 million, and increased to over $850 million in the first three-quarters of 2000. In an effort to stem investor defections from its funds, on October 5, 2000, Liberty announced that it was planning to merge out of existence seventeen of its ninety-five stock and bond funds. The seventeen funds represented $1.7 billion of investor assets. The assets of those seventeen funds were to be merged into ten existing funds in the Liberty family.(8) It is probably safe to assume that the funds that were merged out of existence were the ones with the poorest track records. By merging the funds out of existence Liberty magically made the performance statistics of those funds disappear. The reported returns of the now-merged funds will only contain the live returns of the surviving fund. Of course, the poor returns investors received from the defunct funds did not disappear, it is just that their performance is no longer reported by either the rating services or the funds into which they were merged. (No respect for the dead.) Clearly, future investors are not getting the whole story on the returns earned by investors in the Liberty family of funds.” (RIIIT, pg. 76)

8. Wall Street Journal, October 9, 2000.

Mr. Swedroe then concludes as follows:

“The survivorship bias problem has increased in recent years as mutual fund families try even harder to bury poor performance. In 1998 alone, 387 stock and bond funds were merged out of existence, an increase of forty-three percent over the previous year. An additional 250 funds were liquidated due to investor redemptions. In the first quarter of 1999, the number of vanishing stock funds jumped seventy-four percent.(6) The trend continued into 2000 as a record 735 mutual funds were liquidated or merged. And in the first quarter of 2001, the number of disappearing funds rose thirty-seven percent compared to the year ago period.(7)” (RIIIT, pg. 75-76)

6. Wall Street Journal, May 10, 1999.
7. Barron’s, May 7, 2001.

Incubator Funds: Another Form of Performance Distortion

Another variation of performance distortion derives from the practice of mutual fund companies that utilize so-called “incubator funds.” Mr. Swedroe describes this “performance distortion” phenomenon as follows:

“Incubator funds are newly created funds, seeded by mutual fund families with their own capital. The funds are not available to the public. Here is one way the game may be played. A fund family creates several small-cap funds, possibly even under the same manager. Each fund might own a different group of small-cap stocks. The fund family incubates the funds, safe from public scrutiny. After a few years they bring public only the fund with the best performance. Magically, the performance of the other funds disappears. Unfortunately, a recent SEC ruling allows fund families to report the pre-public performance of incubator funds. Thus we have the potential for huge distortion of reality.

In October 2000, MFS Investment Management of Boston had nineteen funds in incubation, almost all of them less than three years old. If MFS’s prior experience is a good predictor of the future, only one half of these funds will become available to the public; the other half of the currently incubating funds will die a quiet death by liquidation.(9) Which funds do you think will be liquidated? You can readily see how the real performance of MFS’s funds can be distorted without full disclosure.” (RIIIT, pg. 76-77).

9. Wall Street Journal, October 9, 2000.

Summary

In the age of Enron, we should not be surprised to learn of distorted data. That the SEC tolerates these practices in the mutual fund industry is disheartening, but also not altogether surprising. “Be Careful Out There”, said the battle weary sergeant to his police officers in the beginning of each episode of Hill Street Blues. Investors must clearly heed these words of caution when reviewing performance data on mutual funds.

Survivorship bias studies teach us that performance data is subject to distortion, most of it totally legal distortion. Even without accounting for these distortions, the Evidence in favor of index funds, and against actively managed mutual funds, is impressive. When adjustment is made to correct for the major “performance” distortions of taxes, survivorship bias and incubator funds, the argument in favor of index funds is fairly overwhelming.