Why Worry About Survivorship Bias?
For actively managed US equity mutual funds over the period from 1991 to 2020, survivorship bias overstates the median fund alpha by 0.60% per year: The median fund alpha is –0.84% per year among surviving funds compared to –1.44% per year among both surviving and non-surviving funds.
Analyzing the performance of mutual fund managers is popular among financial researchers for the many insights it provides about markets. However, the usefulness of inference from fund returns depends critically on data quality. Because non-surviving funds tend to have poorer performance than surviving funds, studies that examine only surviving strategies suffer from an upward bias in returns known as “survivorship bias.” Our research suggests that survivorship bias overstates the median fund alpha by roughly 50% compared to the survivorship bias-free median (here, alpha refers to performance after accounting for exposures to known drivers of expected returns). It also nearly doubles the proportion of funds that earn a reliably positive alpha.
Impact of Survivorship Bias on Fund Alpha
Survivorship bias matters for two reasons: Funds are often liquidated or merged, and non-surviving funds tend to perform worse than surviving funds. We quantify the effects of survivorship bias using a sample of actively managed US equity mutual funds from the Morningstar Direct database (see the data appendix for the detailed sample-selection criteria). The sample period is January 1991 to June 2020.1
Exhibit 1 shows that, on average over the sample period, about 100 US equity funds were liquidated or merged each year, which is about 5% of the funds available at the beginning of any given year. Disappearances reach a high in 2009, when 275 funds were liquidated or merged, which is 11.5% of the funds in existence at the beginning of 2009.
Disappearing Act
To measure performance, we estimate each fund’s alpha (α) using the Fama/French five-factor model. This allows us to control for differences in performance due to exposures to known drivers of expected returns; namely, the market, size, value, profitability, and investment factors. Each α estimate is based on a fund’s entire history of monthly returns, net of all fees and expenses, during the sample period.2
Studies of mutual fund performance should consider the full distribution of outcomes rather than individual fund results. The large number of funds in the sample (1,557 survivors and 2,545 non-survivors in total) means we are likely to observe extreme α estimates—both positive and negative—by chance alone. As such, we compare the full distribution of α estimates with and without survivorship bias to assess and quantify the impact of the bias on fund alpha.
Exhibit 2 shows the probability distribution of α estimates for all funds as well as separately for surviving and non-surviving funds. Among survivors, the median α estimate is negative: –7 basis points (bps) per month. That is, the median surviving fund underperformed relative to the five-factor model. Including non-surviving funds worsens the picture, as the median α estimate among all funds is –12 bps per month. Survivorship bias thus overstates the median α estimate by 5 bps per month (60 bps per year), or roughly half the magnitude of the survivorship-bias free median.3 The negative median α estimate for all funds is consistent with most studies on active mutual fund performance.4
Survivorship Bias
The performance distribution for non-survivors is to the left of that for survivors (the median for non-survivors is –17 bps per month) and has a more pronounced left tail. This implies that survivorship bias understates the left tail of the performance distribution. For instance, the 1st, 5th, and 10th percentiles are, respectively, –43, –29, and –23 bps per month for surviving funds but –122, –68, and –47 bps per month for all funds.
Turning to the right side of the performance distribution, survivorship bias also overstates the likelihood of a positive investment outcome. Looking only at survivors, 4.5% of funds earn a reliably positive α estimate, i.e., one with a t-statistic above 2. Considering both surviving and non-surviving funds, this proportion is roughly cut in half: Just 2.4% of all funds earn an α estimate that is more than two standard errors above zero, which is below the 3.2% we would expect by chance for this sample.5
What Can We Learn from the Quantification of Survivorship Bias?
Survivorship bias overstates good performance and understates bad outcomes.
Our study highlights the importance of inference based on survivorship-bias free data. For researchers, this means using comprehensive data sets that cover the returns of both surviving and non-surviving strategies.
For investors, our results suggest that one should evaluate any given fund manager based on their full set of funds, including funds that are no longer available for investment—otherwise, a seemingly stellar track record may, in fact, be an incomplete and biased reflection of reality.
Given the noise in security returns and the volatility of premiums, however, past performance is not sufficient to evaluate a fund manager. Researchers and investors should also consider other aspects of the manager, such as their underlying investment philosophy, the robustness in their strategy design, and the efficiency in their portfolio management and trading. All these traits are important in delivering a good investment experience and helping investors achieve their goals.
Learn More
Footnotes
-
1Morningstar Direct records the date of fund liquidations and mergers starting from October 1990. Our research suggests that the database is free from survivorship bias starting from January 1991.
-
2One of our sample selection criteria is that funds must have at least 12 months of returns. This introduces a slight survivorship bias but ensures that we can estimate a fund’s five-factor α.
-
3A rank-sum test rejects that the median α estimate among survivors is the same as that among non-survivors (p = 0.00). The results are similar using means: The mean α estimate is -5 bps per month among survivors but -15 bps per month among all funds and a two-sample t-test with unequal variances rejects that the mean among survivors is the same as that among non-survivors (t = 17.03).
-
4See Carhart (1997), Fama and French (2010), Linnainmaa (2013), and Meyer-Brauns (2016).
-
5Following the methodology of Fama and French (2010), we obtain a by-chance distribution of fund α as the average of 10,000 bootstrapped simulation runs. A simulation run is a random sample (with replacement) of 354 months drawn from the 354 calendar months from January 1991 to June 2020. For each simulation run, we regress, fund by fund, benchmark-adjusted (zero-α) fund returns on the five factors of Fama and French (2015), dropping funds that are in the simulation run for less than 12 months.
references
Carhart, Mark M. 1997. “On Persistence in Mutual Fund Performance,” Journal of Finance 52, no. 1: 57–82.
Fama, Eugene F., and Kenneth French. 2010. “Luck versus Skill in the Cross-Section of Mutual Fund Returns.” Journal of Finance 65, no. 5: 1915–1947.
Fama, Eugene F., and Kenneth French. 2015. “A Five-Factor Asset Pricing Model.” Journal of Financial Economics 116, no. 11: 1–22.
Linnainmaa, Juhani T. 2013. “Reverse Survivorship Bias.” Journal of Finance 68, no. 3: 789–813.
Meyer-Brauns, Philipp. 2016.
“Mutual Fund Performance through a Five-Factor Lens.” Dimensional Fund Advisors white paper.
data appendix
The sample is actively managed, US-domiciled, USD-denominated, US-equity invested, open-end mutual funds on the Morningstar Direct database, excluding index funds and fund-of-funds. Funds are included in the sample upon first passing an assets-under-management (AUM) threshold of $5 million measured in June 2020 US dollars. To mitigate incubation bias, we exclude funds with a first monthly return within the last 5 years of the sample period. We also exclude funds with fewer than 12 months of returns during the sample period. Once a fund is included in the sample, we keep it until it is either liquidated or merged or we reach June 2020. Surviving funds are those still available at the end of June 2020.
Glossary
Active management: A portfolio management approach that aims to outperform a market rate or return, or a specific benchmark, by choosing investments that deviate from the market portfolio or benchmark.
Survivorship bias: Looking only at live funds and ignoring those that have liquidated or merged.
Median: The value below which 50% of the outcomes in a sample are found. Equivalently, 50% of the outcomes are found above the median.
Alpha: The rate of return on an investment in excess of a benchmark or the return predicted by a financial model. A more positive alpha implies greater outperformance, while a more negative alpha implies greater underperformance.
Surviving fund: A fund still available at the end of the sample period (June 2020).
Liquidated fund: A fund that sells all its assets and ceases to exist altogether.
Merged fund: A fund that ceases to exist as a separate entity because it is combined with other funds.
Fama/French Five-Factor Model: A financial model describing an equity security’s expected return in excess of the return on short-term bills as a linear function of five explanatory variables or “factors.” The factors are the market’s excess return over short-term bills (the market factor), the return difference between small and large stocks (the size factor), the return difference between value and growth stocks (the value factor), the return difference between high and low profitability stocks (the profitability factor), and the return difference between low and high asset-growth stocks (the investment factor).
Small and large stocks: Small stocks are those with a relatively low market capitalization, while large stocks are those with relatively large market capitalization.
Value and growth stocks: Value stocks are those with relatively low prices relative to book value, while growth stocks are those with relatively high prices relative to book value.
Profitability: A company’s operating income before depreciation and amortization minus interest expense scaled by book equity.
Investment or asset growth: A company’s annual percentage change in total book assets.
Probability distribution: A tabulation or graphing of the frequency of different outcomes in a sample.
Percentile: The value below which a given percentage of outcomes in a sample are found. For example, the 5th percentile is the value below which 5% of the outcomes are found; equivalently, 95% of the outcomes are found above the 5th percentile.
Tails of a probability distribution: The extreme outcomes in a sample that occur with a relatively low frequency.
t-Statistic: A measure of the statistical reliability of an estimate. It is defined as the value of the estimate divided by the standard error of the estimate, where the standard error is a measure of the imprecision in the estimate. For example, a value more positive than 2 or more negative than -2 indicates that the estimate is more than two standard errors away from zero.
Disclosures
Eugene Fama and Ken French are members of the Board of Directors of the general partner of, and provide consulting services to, DFAL and DIL.
FOR PROFESSIONAL USE ONLY. NOT FOR USE WITH RETAIL INVESTORS OR THE PUBLIC.
The information in this material is intended for the recipient’s background information and use only. It is provided in good faith and without any warranty or representation as to accuracy or completeness. Information and opinions presented in this material have been obtained or derived from sources believed by Dimensional to be reliable and Dimensional has reasonable grounds to believe that all factual information herein is true as at the date of this document. It does not constitute investment advice, recommendation, or an offer of any services or products for sale and is not intended to provide a sufficient basis on which to make an investment decision. It is the responsibility of any persons wishing to make a purchase to inform themselves of and observe all applicable laws and regulations. Unauthorized reproduction or transmitting of this material is strictly prohibited. Dimensional accepts no responsibility for loss arising from the use of the information contained herein.
“Dimensional” refers to the Dimensional separate but affiliated entities generally, rather than to one particular entity. These entities are Dimensional Fund Advisors LP, Dimensional Fund Advisors Ltd., DFA Australia Limited, Dimensional Fund Advisors Canada ULC, Dimensional Fund Advisors Pte. Ltd., Dimensional Ireland Limited, Dimensional Japan Ltd., and Dimensional Hong Kong Limited. Dimensional Hong Kong Limited is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
WHERE ISSUED BY DIMENSIONAL IRELAND LIMITED
Issued by Dimensional Ireland Limited (Dimensional Ireland), with registered office 3 Dublin Landings, North Wall Quay, Dublin 1, Ireland. Dimensional Ireland is regulated by the Central Bank of Ireland (Registration No. C185067). Directed only at professional clients within the meaning of Markets in Financial Instruments Directive (MiFID) (2014/65/EU). Information and opinions presented in this material have been obtained or derived from sources believed by Dimensional Ireland to be reliable, and Dimensional Ireland has reasonable grounds to believe that all factual information herein is true as at the date of this document.
Dimensional Ireland issues information and materials in English and may also issue information and materials in certain other languages. The recipient’s continued acceptance of information and materials from Dimensional Ireland will constitute the recipient’s consent to be provided with such information and materials, where relevant, in more than one language.
WHERE ISSUED BY DIMENSIONAL FUND ADVISORS LTD.
Issued by Dimensional Fund Advisors Ltd. (Dimensional UK), 20 Triton Street, Regent’s Place, London, NW1 3BF. Dimensional UK is authorised and regulated by the Financial Conduct Authority (FCA). Directed only at professional clients as defined by the rules of the FCA. Information and opinions presented in this material have been obtained or derived from sources believed by DFAL to be reliable, and Dimensional UK has reasonable grounds to believe that all factual information herein is true as at the date of this document.
Dimensional UK issues information and materials in English and may also issue information and materials in certain other languages. The recipient’s continued acceptance of information and materials from Dimensional UK will constitute the recipient’s consent to be provided with such information and materials, where relevant, in more than one language.
NOTICE TO INVESTORS IN SWITZERLAND: This is an advertising document.
RISKS
Investments involve risks. The investment return and principal value of an investment may fluctuate so that an investor’s shares, when redeemed, may be worth more or less than their original value. Past performance is not a guarantee of future results. There is no guarantee strategies will be successful.