Systematically Evaluating Systematic Managers
-
Systematic active investing can be more reliable and less costly than a traditional active approach while offering diversification and ease of monitoring.
-
Investors should evaluate a systematic manager’s research, design, process, and track record before hiring.
-
After hiring, systematic managers should be evaluated for their premium capture, risk management, and total costs.
Systematic investing is a rules-based approach to managing money. It replaces the subjectivity of traditional active management with quantitative research and avoids the rigidities of indexing through an active but process-driven pursuit of premiums.1 If done well, a systematic active approach can be more reliable and less costly than a traditional active approach without sacrificing the greater diversification and easier monitoring typical of indexing.
Many asset owners, consultants, and wealth advisors now recognize that a systematic active approach is distinct from the various forms of indexing and discretionary management, including those adopting a more quantitative approach. One reason for the growing interest in systematic strategies is that they can work well on their own or alongside other strategies in an asset allocation. Common examples include using a systematic core portfolio in a core-satellite allocation or using a systematic style portfolio as a completion (see Exhibit 1). The other portfolios can be traditional active, indexed, or alternative, including private.
Using a Systematic Approach
Examples of asset allocations incorporating systematic strategies
Investors considering systematic strategies face an important choice among a multitude of managers who differ on everything from investment philosophy to trading approach. How should investors differentiate between systematic managers before hiring? After hiring, what are reasonable criteria for assessing whether a manager has delivered on what was promised? Here, we provide a systematic framework for evaluating systematic managers and the strategies they offer, both before and after hiring.2
Before Hiring
Before Hiring
Overview of criteria for evaluating systematic managers before hiring
Research
Systematic investing has its roots in academic, factor-based asset pricing models.3 The proliferation of factors, however, means managers must research which premiums are worth pursuing and how. At the same time, there is no shortage of incentives to uncover seemingly useful patterns in the data. A good manager has the skill to distinguish signal from noise and the discipline to avoid constant tweaks.
When evaluating the premiums a manager pursues, start with economic rationale, then look for statistical reliability, robustness, and evidence of real-world efficacy. Be mindful of a black-box approach, redundant factors, and excessive turnover.4 Ask about factors no longer on the manager’s list, and why. Managers claiming to find new factors or “alpha signals” should be asked, “relative to what?”5
Systematic managers commonly use simulations to demonstrate a strategy’s viability. While potentially helpful in setting reasonable expectations, simulations come with many degrees of freedom and are prone to producing backtested performance unattainable in practice. The results can be highly sensitive to the time period, investable universe, weighting scheme, rebalancing choices, and cost assumptions.6 Look for a transparent methodology that adheres to well-established standards and realistic, justifiable assumptions.7
Design
Translating research into investable strategies requires balancing tradeoffs. For a systematic manager, portfolio design should balance pursuing premiums with managing risks and controlling costs. Look for a direct link to the research and clear justifications for the choices around security selection, weighting scheme, and refinements. Some return drivers are better suited for informing exclusions or trading decisions rather than asset allocation, and a good design makes the most of this distinction.8 A good design also accounts for premium interactions and controls concentrations and turnover.9 Ask about the design’s expected capacity and potential for customization.
Process
A key advantage of a systematic active approach relative to indexing is the potential for a continuous implementation process. The infrequent and prescheduled rebalancing that comes with indexing implies style drift between rebalances and price pressure around index reconstitutions.10A continuous process can avoid such inefficiencies and instead incrementally rebalance portfolios daily as prices change, characteristics are updated, and cash comes in or out of the portfolio. A good process also cost effectively incorporates short-lived return drivers, responds efficiently to corporate actions, and spreads turnover across all trading days of the year.
Portfolio management at a systematic firm should aim to maintain the strategies positioned according to their designs while managing risks and maximizing the value of holdings. This requires infrastructure for data management, order generation, and securities lending, as well as the ability to handle corporate actions, engage in investment stewardship, and dynamically assess risks. Outsourcing parts of portfolio management may reduce infrastructure and operational costs but result in a less integrated, more rigid process.
How systematic managers trade can make or break their ability to deliver premiums. The work put into research, design, and portfolio management is all for naught if given up in trading costs. Trading incurs explicit costs (such as commissions and stamp duties) and implicit costs (such as price impact), and executing at favorable prices may require flexibility in the sizing, timing, and routing of orders. While markets have become more liquid, liquidity varies across securities and regions and can quickly evaporate during volatile times. A good manager has trading in focus throughout the implementation process, including conducting research into market microstructure and designing strategies with built-in flexibilities.
Track Record
A track record of outperformance in a particular strategy is not definitive proof of skill, especially when viewed in isolation. Most managers will have a few strategies that beat their benchmarks at a given time, and outperformance rates can be inflated by closing underperforming strategies or merging them with better-performing ones. When evaluating a manager’s track record, look for high survival and outperformance rates for the manager’s full range of strategies, both over the long term and across asset classes. Compare the manager’s track record to that of peers or the industry (see The Fund Landscape or the SPIVA Scorecard). Ask about the reasons for strategy mergers and closures.
After Hiring
After hiring, evaluate a systematic manager on premium capture, risk management, and total costs. Exhibit 3 gives an overview. Below, we elaborate on some key points.
Premium Capture
It is tempting to look at a strategy’s unconditional performance for clues about the manager’s implementation skill. But the volatility of returns means this inference is noisy.11 Luck, both good and bad, can drive performance, especially over short periods.12 It is more informative to assess a strategy’s positioning and premium capture.
When assessing positioning, avoid overreliance on factor regressions. While potentially useful in gauging sensitivities, factor regressions impose restrictive assumptions (such as linearity and constant factor loadings) and may be sensitive to the choice of factors and the time period. Analyzing holdings and characteristics provides a more robust, period-by-period look under the hood without estimation error. A well-positioned strategy has a clear and consistent focus on securities expected to deliver the premiums it pursues. Multidimensional grids of portfolio weights relative to market weights are also useful in assessing how the strategy accounts for premium interactions.
When assessing premium capture, look at a strategy’s performance relative to its benchmark, conditional on how the premiums materialized.13 A well-managed strategy will on average beat its benchmark net of fees during periods when the premiums it pursues are positive.
Risk Management
A good systematic manager is effective at both delivering premiums and managing risks. One aspect of effective risk management is broad diversification across securities, issuers, sectors, and countries (when applicable), as this can mitigate the impact of adverse idiosyncratic shocks. Another is an agile process that incorporates real-time market data (such as prices, trading volumes, and credit spreads), as well as human oversight to handle, for instance, geopolitical events.14 Look for a manager that demonstrates both.
Total Costs
Trading costs can be a major drag on premium capture and investors’ net returns. Transaction-cost analyses (TCA) should be part of the routine monitoring of managers. Look for a sizable price advantage relative to multiple benchmarks, including adjacent trades (before and after) and liquidity demand (selling at the bid and buying at the offer, or SBBO).15 A good manager also consistently ranks favorably relative to peers on both explicit and implicit trading costs.
The total cost of ownership encompasses not just the expense ratio but also foregone returns and incurred monitoring costs. Foregone returns are an opportunity cost for investors. Common examples are cash drag, premiums left on the table due to style drift, and profits from securities lending kept by the manager or not earned. Monitoring costs include the time and effort an investor spends understanding the manager’s choices as well as the strategy’s positioning and performance. A cost-conscious manager looks to mitigate all costs.
After Hiring
Overview of criteria for evaluating systematic managers after hiring
Takeaways
Systematic investing has the potential to add value through an active, yet disciplined, pursuit of premiums rooted in quantitative research and efficient implementation. Whether that potential is realized depends largely on the choice of manager. We hope the framework outlined here will help investors systematically evaluate systematic managers.
Footnotes
- 1Systematic investing also goes by the names quantitative investing and factor-based investing.
- 2Dimensional is a systematic active manager. We are acutely aware that writing this article resembles handing in one’s homework along with a description of how it should be graded. Still, the experience we have amassed over more than 40 years of managing systematic strategies should give credibility to the criteria we propose. Our hope is they are applied just as strictly to us as to any other systematic manager. In fact, most of the criteria apply beyond systematic managers.
- 3Robert C. Merton, “An Intertemporal Capital Asset Pricing Model,” Econometrica 41, no. 5 (1973): 867–887.
- 4Marlena I. Lee, Savina Rizova, and Samuel Yusun Wang, “The Cross-Section of Global Corporate Bond Returns” (research paper, Dimensional Fund Advisors, March 2022); also available on SSRN.
- 5Eugene F. Fama and Kenneth R. French, “Dissecting Anomalies with a Five-Factor Model,” Review of Financial Studies 29, no. 1 (2016): 69–103.
- 6Wei Dai and Namiko Saito, “Weighting for the Right One: Weighting Scheme Design for Systematic Equity Portfolios” (research paper, Dimensional Fund Advisors, February 2022); also available on SSRN. Andrew Detzel, Robert Novy-Marx, and Mihail Velikov, “Model Comparison with Transaction Costs,” Journal of Finance 78, no. 3 (2023): 1743–1775.
- 7Robert Novy-Marx and Mihail Velikov, “Betting Against Betting Against Beta,” Journal of Financial Economics 143, no. 1 (2022): 80–106.
- 8Savina Rizova and Namiko Saito, “Investment and Expected Stock Returns” (research paper, Dimensional Fund Advisors, July 2020); also available on SSRN. Wei Dai, Mamdouh Medhat, Robert Novy-Marx, and Savina Rizova, “” (research paper, Dimensional Fund Advisors, January 2023); also available on SSRN.
- 9Wei Dai, Namiko Saito, and Stephen Watson, “Pursuing Multiple Premiums: Combination vs. Integration” (research paper, Dimensional Fund Advisors, February 2021); also available on SSRN.
- 10Wes Crill, “Out of Bounds: Style Drift in the Russell 2000 Value Index,” Insights (blog), Dimensional Fund Advisors, June 23, 2021; Kaitlin Hendrix and Trey Roberts, “Beware the Hidden Costs of Indexing,” Insights (blog), Dimensional Fund Advisors, June 23, 2022.
- 11Eugene F. Fama and Kenneth R. French, “Volatility Lessons,” Financial Analyst Journal 74, no. 3 (2018): 43–53.
- 12Eugene F. Fama and Kenneth R. French, “Luck versus Skill in the Cross-Section of Mutual Fund Returns,” Journal of Finance 65, no. 5 (2010): 1915–1947.
- 13Wes Crill, “Expectations vs. Reality in Value Funds,” Insights (blog), Dimensional Fund Advisors, March 3, 2023.
- 14Wei Dai, Alan Hutchison, and Samuel Yusun Wang, “Credit Spreads, Rating Downgrades, and Downside Performance: A Market-Informed Approach to Monitoring Credit Risk” (research paper, Dimensional Fund Advisors, March 2020); also available on SSRN. Karen Umland, “Navigating Geopolitical Events,” Insights (blog), March 1, 2022.
- 15Jerry Liu, “The Numbers Are In: Flexible Trading Helps Across Markets,” Insights (blog), July 1, 2021.
Disclosures
Eugene Fama and Ken French are members of the Board of Directors of the general partner of, and provide consulting services to, Dimensional Fund Advisors LP. Robert Merton provides consulting services to Dimensional Fund Advisors LP. Robert Novy-Marx provides consulting services to Dimensional Fund Advisors LP.
The following persons listed as references are employees of Dimensional Investment LLC, a subsidiary of Dimensional Fund Advisors LP: Marlena Lee, Savina Rizova, Samuel Wang, Wei Dai, Namiko Saito, Mamdouh Medhat, Wes Crill, Kaitlin Hendrix, Trey Roberts, Alan Hutchison, Karen Umland, and Jerry Liu.
FOR PROFESSIONAL USE ONLY. NOT FOR USE WITH RETAIL INVESTORS OR THE PUBLIC.
The information in this material is intended for the recipient’s background information and use only. It is provided in good faith and without any warranty or representation as to accuracy or completeness. Information and opinions presented in this material have been obtained or derived from sources believed by Dimensional to be reliable, and Dimensional has reasonable grounds to believe that all factual information herein is true as at the date of this material. It does not constitute investment advice, a recommendation, or an offer of any services or products for sale and is not intended to provide a sufficient basis on which to make an investment decision. Before acting on any information in this document, you should consider whether it is appropriate for your particular circumstances and, if appropriate, seek professional advice. It is the responsibility of any persons wishing to make a purchase to inform themselves of and observe all applicable laws and regulations. Unauthorized reproduction or transmission of this material is strictly prohibited. Dimensional accepts no responsibility for loss arising from the use of the information contained herein.
This material is not directed at any person in any jurisdiction where the availability of this material is prohibited or would subject Dimensional or its products or services to any registration, licensing, or other such legal requirements within the jurisdiction.
“Dimensional” refers to the Dimensional separate but affiliated entities generally, rather than to one particular entity. These entities are Dimensional Fund Advisors LP, Dimensional Fund Advisors Ltd., Dimensional Ireland Limited, DFA Australia Limited, Dimensional Fund Advisors Canada ULC, Dimensional Fund Advisors Pte. Ltd., Dimensional Japan Ltd., and Dimensional Hong Kong Limited. Dimensional Hong Kong Limited is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
RISKS
Investments involve risks. The investment return and principal value of an investment may fluctuate so that an investor’s shares, when redeemed, may be worth more or less than their original value. Past performance is not a guarantee of future results. There is no guarantee strategies will be successful.
UNITED STATES
This information is provided for registered investment advisors and institutional investors and is not intended for public use. Dimensional Fund Advisors LP is an investment advisor registered with the Securities and Exchange Commission.
CANADA
This material is issued by Dimensional Fund Advisors Canada ULC for advisors, dealers, and institutional investors and is not intended for public use. The other Dimensional entities referenced herein are not registered resident investment fund managers or portfolio managers in Canada.
This material is not intended for Quebec residents.
Commissions, trailing commissions, management fees, and expenses all may be associated with mutual fund investments. Please read the prospectus before investing. Unless otherwise noted, any indicated total rates of return reflect the historical annual compounded total returns, including changes in share or unit value and reinvestment of all dividends or other distributions, and do not take into account sales, redemption, distribution, or optional charges or income taxes payable by any security holder that would have reduced returns. Mutual funds are not guaranteed, their values change frequently, and past performance may not be repeated.
AUSTRALIA
In Australia, this material is provided by DFA Australia Limited (AFSL 238093, ABN 46 065 937 671). It is provided for financial advisors and wholesale investors for information only and is not intended for public use. No account has been taken of the objectives, financial situation or needs of any particular person. Accordingly, to the extent this material constitutes general financial product advice, investors should, before acting on the advice, consider the appropriateness of the advice, having regard to the investor’s objectives, financial situation and needs.
NEW ZEALAND
This material has been prepared and provided in New Zealand by DFA Australia Limited, (incorporated in Australia, AFS License No.238093, ABN 46 065 937 671). This material is provided for financial advisers only and is not intended for public use. All material that DFA Australia Limited provides has been prepared for advisers, institutional investors and clients who are classified as Wholesale investors under the Financial Markets Conduct Act 2013. This material does not give any recommendation or opinion to acquire any financial advice product, and is not financial advice to you or any other person.
WHERE ISSUED BY DIMENSIONAL IRELAND LIMITED
Issued by Dimensional Ireland Limited (Dimensional Ireland), with registered office 3 Dublin Landings, North Wall Quay, Dublin 1, Ireland. Dimensional Ireland is regulated by the Central Bank of Ireland (Registration No. C185067).
Directed only at professional clients within the meaning of Markets in Financial Instruments Directive (MiFID) (2014/65/EU).
WHERE ISSUED BY DIMENSIONAL FUND ADVISORS LTD.
Issued by Dimensional Fund Advisors Ltd. (Dimensional UK), 20 Triton Street, Regent’s Place, London, NW1 3BF. Dimensional UK is authorised and regulated by the Financial Conduct Authority (FCA) - Firm Reference No. 150100.
Directed only at professional clients as defined by the rules of the FCA.
Dimensional UK and Dimensional Ireland issue information and materials in English and may also issue information and materials in certain other languages. The recipient’s continued acceptance of information and materials from Dimensional UK and Dimensional Ireland will constitute the recipient’s consent to be provided with such information and materials, where relevant, in more than one language.
NOTICE TO INVESTORS IN SWITZERLAND: This is advertising material.
JAPAN
For Institutional Investors only.
This material is deemed to be issued by Dimensional Japan Ltd., which is regulated by the Financial Services Agency of Japan and is registered as a Financial Instruments Firm conducting Investment Management Business and Investment Advisory and Agency Business.
Dimensional Japan Ltd.
Director of Kanto Local Finance Bureau (FIBO) No. 2683
Membership: Japan Investment Advisers Association
FOR LICENSED OR EXEMPT FINANCIAL ADVISORS AND INSTITUTIONAL INVESTORS IN SINGAPORE
This material is deemed to be issued by Dimensional Fund Advisors Pte. Ltd. (UEN:201210847M), which is regulated by the Monetary Authority of Singapore and holds a capital markets services license for fund management.
This material is not an advertisement, has not been reviewed by the Monetary Authority of Singapore or the Central Provident Fund (CPF) Board, and should not be shown to prospective retail investors.
For use by institutional investors and licensed or exempt financial advisors only in Singapore for internal training and educational purposes and not for the purpose of inducing, or attempting to induce, such institutional investors or financial advisors to make an investment. Not for use with the public.
FOR LICENSED FINANCIAL ADVISORS AND INSTITUTIONAL INVESTORS IN HONG KONG
This material is deemed to be issued by Dimensional Hong Kong Limited (CE No. BJE760), which is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
For use by licensed financial advisors and institutional investors who are “professional investors” (as defined in the Securities and Futures Ordinance [Chapter 571 of the Laws of Hong Kong] and its subsidiary legislation) only in Hong Kong. This material is provided solely for internal training and educational purposes and is not for the purpose of inducing, or attempting to induce, such financial advisors and institutional investors to make an investment nor for the purpose of providing investment advice. Not for use with the public. This material is not intended to constitute and does not constitute marketing of the services of Dimensional Hong Kong or its affiliates to the public of Hong Kong.
Financial advisors in Hong Kong shall not actively market the services of Dimensional Hong Kong Limited or its affiliates to the Hong Kong public.