Back-tests

Ideally back-testing is the method of feeding historical data into a purely quantitative investment system in order to simulate the investment performance the system would have if it had been used in a real world investment fund with real asset transactions. However, in practice many back-tests are often far from this ideal and instead produce very crude approximations of what the quantitative investment system could do in a real world investment fund.

Unfortunately, inadequate and poorly performed back-testing by some investment professionals has given back-testing a bad reputation. However, if it is done right it can be as good evidence of potential future investment performance as a real world investment track-record. Back-tests can even be better at this than a real world track-record because they can often test an investment system’s performance over significantly more years and thereby include several more investment return cycles than possible in most real world funds. This is so, because real world funds often have only existed in a few years or because their investment managers or investment strategy has changed within the last few years.

The back-tests that are made to validate the investment performance of Alpha Captech’s quantitative investment system are state-of-the-art because:

I) Portfolio replicating 1 to 1 back-tests

Alpha Captech applies what we call 1 to 1 back-tests meaning the live fund portfolio will always be a nearly identical copy of the latest back-test portfolio. In other words, the investment composition of the live fund will be generated by trading the differences to the latest available composition of the back-test portfolio. These portfolios are frequently made to update the back-tests and to know what to trade in the live fund. Alpha Captech calls it a 1 to 1 back-test in order to stress that our back-tests are so realistic that they can be used directly to govern the composition of a real world investment fund. Needless to say, that if a back-test is so realistic that it can be used directly to govern the investment portfolio of a live fund the historical performance data from such a back-test will also be far more realistic than those obtained from other types of back-tests that are not actionable in this manner because of deep compromises and simplifications made in their back-test procedures.

II) Testing performance for a given level of investment capacity to 1 back-tests

All quantitative investment systems will perform worse the more capital you invest using the specific strategy. This is so because quantitative systems rank stocks from best to worst and as the portfolio size grows the system will necessarily have to include more of less highly ranked stocks. Therefore, in order to obtain the most realistic performance estimates you need to back-test the quant system for each level of asset under management that the intended investment fund is expected to handle. Unfortunately, many back-tests are often performed not on actual USD portfolio sizes but rather on a certain percentile of the probed investment universe with the highest signal score made by the quantitative system. Such a percentile often contains more companies than it is practical to invest. Moreover, the stock returns in these back-tests are typically measured as either equally weighted returns or as market cap weighted returns neither of which can meaningfully be obtained in a real world investment fund. Back-tests based on percentile portfolios are OK for academic studies that just need to verify whether an anomaly exist or not. However, investment professionals should always test their quant systems on portfolios that are specified for a given investment level and calculate returns that are weighted with the USD value of each position in these portfolios.

III) Back-tested on data since 1992

In order for a back-test to be a good estimator of potential future investment performance it needs to be long enough to cover several investment return cycles. Many of the financial market anomalies that are exploited in the system that Alpha Captech has developed exhibit investment return cycles that are decades long meaning they can perform extremely well for ten years and another ten years they just beat the market a little. Obviously you need the average performance over several such 10 years cycles in order to get a good estimator for potential future performance.

IV) Developed system not based on estimated parameters that fit data in a certain time frame and thus may only work well in that time frame

This implies that out-of-sample back-tests are not relevant for the system that Alpha Captech has developed. It also means that the investment performance of Alpha Captech’s system is highly suited to be proven in back-tests using historical data and much more so than the time frame dependent parametric systems.

V) Deducting the costs of implementing the investments

All investments strategies have costs that need to be deducted from the gross returns the back-test portfolios generate. These costs can be direct cost related to trading, fund administration, depository and custodian services, distribution/customer acquisition, investment management, etc. However, they can also be indirect cost such as expected adverse price effects when one has to trade the portfolio positions. Adverse price effects are also generated when other anomaly investors are trying to buy and sell more or less the same kind of stocks at the same time. Alpha Captech always calculate two back-test scenarios: a main back-test with typical costs and a stress back-test with extraordinary high costs. This is an effective way of showing how sensitive the various performance measures calculated for the back-tested system are to changes in the cost assumptions. Moreover, Alpha Captech calculate its back-tested performance measures after performance fees in a simulation model that is 100% accurate to how that calculation is done in Alpha Captech’s investment fund.

VI) Entire system is tested. Not just a partial test

The quantitative investment strategy applied by Alpha Captech can be back-tested in its entirety. It is a long-only strategy in exchange traded stocks only so we have all the data needed to do a complete back-test of the entire system. Many other quantitative investment strategies are dependent on shorted assets and complex derivatives that cannot be back-tested because of a lack of needed historical price data and implementation cost for that part of their strategy. So they can only partially back-test their strategy for the part of the system that have the needed data. Obviously partially back-tested systems produce much less meaningful performance data.

VII) Rebalancing on a monthly basis using overlapping portfolios

The back-tests used to validate the developed system are rebalanced on a monthly basis meaning the quant system is fed with new and updated data each month in order to generate a new and up to date investment portfolio. A complicating matter in this process is that the different financial anomalies that the system exploits have different times they need to stay in the portfolio before the system will recalculate whether they should stay for another holding period or be sold off. That holding period is longer than one month for all the applied anomalies. Therefore, a system of overlapping portfolios are used with sub portfolios for each anomaly being created each month and with its positions being hold for the intended period. These sub portfolios are subsequently merged into the full investment portfolio.

VIII) No unrealistic assumptions regarding the ability to use shorted portfolio assets

Alpha Captech does not use any shorted assets in its investment strategy and that makes the back-tests for the developed system more credible than back-tests that assume that all assets in their investment universe can be shorted at will at zero costs. The latter is often assumed in academic studies. The problem with this assumption about being able to short any asset at zero cost is that many assets will not have a market for shorted positions and even if a short contract can be created it may be at very high costs. As a result the back-test portfolios with shorted positions can either not be traded in the real world or they can only be traded at considerable transaction costs. Needless to say that both of these issues decreases the quality of the estimated investment performance produced by the back-test for a particular investment strategy that use shorted assets.

IX) Numerous robustness tests carried out

The scientific method requires that any test done to validate anything is followed up with numerous of robustness tests in order to verify that the original test results were not a consequence of erroneous test procedures, data errors, peculiar test conditions or any other reasons that one can imagine might influence the results in non-intended ways. Alpha Captech has carried out numerous of robustness test for the developed system or the previous versions of the developed system including but not limited to:

  1. Testing different models for different anomalies
  2. Testing different auxiliary selection models not directly related to financial market anomalies
  3. Testing different back-test procedures
  4. Testing different measurement of returns. For instance, different data sources or returns calculated by local currency or by USD only
  5. Testing different company sizes as measured by the market capitalization
  6. Testing different regions. For instance, EU27, USA and Japan
  7. Testing different industry selections, such as, financials and non-financials
  8. Testing different stock trading periods, such as, trading portfolios in January or trading them in July
  9. Testing different stock trading rules. For instance, different trading rules have been used to test the system at different capacities. E.g. $1.8B and $12B portfolios. System also tested different ways of measuring liquidity e.g. 5 days or 1MD
  10. Testing the system using different stock holding periods, such as, buy and hold the stocks in the portfolios for 1 year or alternatively for 2 years or 3 months
  11. Testing the system using different back-testing periods, such as from 1992 to 1999, from 2000 to 2007 and from 2008 to 2018