Home
Team
Product
Setup
Objective
Careers
Investors

Contact
Back-tests - Not all are created equal

Ideally back-testing is the method of feeding historical data into a purely quantitative investment system in order to simulate the investment performance the system would have if it had been used in a real world investment fund with real asset transactions. In practice, many back-tests are often far from this ideal and instead produce very crude approximations of what the systems could do in a real world investment fund.

Unfortunately, inadequate back-testing by some practitioners has given back-testing a bad reputation. However, if it is done right it can be as good evidence of potential future investment performance as a real world investment track-record. Back-tests can even be better at this than a real world track-record because they can often test an investment system's performance over significantly more years and thereby include several more investment return cycles than possible in most real world funds. This is so, because real world funds have often only existed in a few years or because their investment managers or investment strategy has changed within the last few years.

The back-tests that are made to prove the investment performance of the quant system used by Alpha Captech are state-of-the-art because:

  1. Portfolio replicating 1 to 1 back-tests
    Alpha Captech applies what we call 1 to 1 back-tests meaning the live fund portfolio will always be a nearly identical copy of the latest back-test portfolio. In other words, the investment composition of the live fund will be generated by trading the differences to the latest available composition of the back-test portfolio. These portfolios are frequently made to update the back-tests and to know what to trade in the live fund. Alpha Captech calls it a 1 to 1 back-test in order to stress that our back-tests are so realistic that they can be used directly to govern the composition of a real world investment fund. Needless to say, that if a back-test is so realistic that it can be used directly to govern the investment portfolio of a live fund the historical performance data from such a back-test will also be far more realistic than those obtained from other types of back-tests that are not actionable in this manner because of deep compromises and simplifications made in their back-test procedures.
     
  2. Testing performance for a given level of investment capacity
    All quantitative investment systems will perform worse the more capital you invest using the specific strategy. Therefore, in order to obtain the most realistic performance estimates you need to back-test the quant system for each level of asset under management that the intended investment fund is expected to handle. Unfortunately, many back-tests are often performed not on actual USD portfolio sizes but rather on a certain percentile of the probed investment universe with the highest signal score made by the quantitative system. Such a percentile often contains more companies than it is practical to invest. Moreover, the stock returns in these back-tests are typically measured as either equally weighted returns or as market cap weighted returns neither of which can meaningfully be obtained in a real world fund. Back-tests based on percentile portfolios are OK for academic studies that just need to verify whether an anomaly exist or not. However, investment professionals should always test their quant systems on portfolios that are specified for a given investment level and calculate returns that are weighted with the USD value of each position in these portfolios.
     
  3. Back-tested on data since 1992
    In order for a back-test to be a good estimator of potential future investment performance it needs to be long enough to cover several investment return cycles. Many of the financial market anomalies that are exploited in the system that Alpha Captech uses exhibit investment return cycles that are decades long meaning they can perform extremely well for ten years and another ten years they just beat the market a little. Obviously you need the average performance over several such cycles in order to get a good estimator for potential future performance.
     
  4. Not based on estimated parameters that fit data in a certain time frame and thus may only work well in that time frame
    This implies that out-of-sample back-tests are not relevant for the system that Alpha Captech uses. It also means that the investment performance of the developed system is highly suited to be proven in back-tests using historical data and more so than the time frame dependent parametric systems.
     
  5. Deducting the costs of implementing the investments
    All investments strategies have costs that need to be deducted from the gross returns the back-test portfolios generate. These costs can be direct cost related to trading, fund administration, depository and custodian services, distribution, investment management, etc. However, they can also be indirect cost such as expected adverse price effects when one has to trade the portfolio positions. Alpha Captech always calculate two back-test scenarios: a main back-test with typical costs and a stress back-test with extraordinary high costs. This is an effective way of showing how sensitive the various performance measures calculated for the back-tested system is to changes in the cost assumptions.
     
  6. Entire system is tested (not a partial test)
    The quantitative investment strategy applied by Alpha Captech can be back-tested in its entirety. It is a long-only strategy in exchange traded stocks only so we have all the data needed to do a complete back-test of the entire system. Many other quantitative investment strategies are dependent on shorted assets and complex derivatives that cannot be back-tested because of a lack of needed historical price data and implementation cost for that part of their strategy. So they can only partially back-test their strategy for the part of the system that have the needed data. Obviously partially back-tested systems produce much less meaningful performance data.
     
  7. Rebalancing on a monthly basis using overlapping portfolios
    The back-tests are rebalanced on a monthly basis meaning the quant system is fed with new and updated data each month in order to generate a new and up to date investment portfolio. A complicating matter in this process is that the different financial anomalies that the system exploits have different times they need to stay in the portfolio before the system will recalculate whether they should stay for another holding period or be sold off. That holding period is longer than one month for all the applied anomalies. Therefore, a system of overlapping portfolios are used with sub portfolios for each anomaly being created each month and with its positions being hold for the intended period. These sub portfolios are subsequently merged into the full investment portfolio.
     
  8. No unrealistic assumptions regarding the ability to use shorted portfolio assets
    Alpha Captech does not use any shorted assets in its investment strategy and that makes the back-tests for the developed system more credible than back-tests that assume that all assets in their investment universe can be shorted at will at zero costs. The latter is often assumed in academic studies. The problem with this assumption about being able to short any asset at zero cost is that many assets will not have a market for shorted positions and even if a short contract can be created it may be at very high costs. As a result the back-test portfolios with shorted positions can either not be traded in the real world or they can only be traded at considerable transaction costs.  Needless to say that both of these issues decreases the quality of the estimated investment performance produced by the back-test for a particular investment strategy that use shorted assets.
     
  9. Numerous robustness tests carried out
    The scientific method requires that any test done to prove anything is followed up with numerous of robustness tests in order to verify that the original test results were not a consequence of erroneous test procedures, data errors, peculiar test conditions or any other reasons that one can imagine might influence the results in non-intended ways. Mathiesen has carried out numerous of robustness test for the developed system or the previous versions of the developed system including:
     
    1. Testing the system using different models for the anomalies
    2. Testing the system using different auxiliary selection models not directly related to financial market anomalies
    3. Testing the system using different back-test procedures
    4. Testing the system using different measurement of returns. For instance, different data sources or returns calculated by local currency or by USD only
    5. Testing the system on different company sizes as measured by the market capitalization
    6. Testing the system on different regions. For instance, EU27, USA and Japan
    7. Testing the system on different industry selections, such as, financials and non-financials
    8. Testing the system using different stock trading periods, such as, trading portfolios in January or trading them in July
    9. Testing the system using different stock trading rules. For instance, different trading rules have been used to test the system at different capacities. E.g. USD 1.2B and USD 12B portfolios
    10. Testing the system using different stock holding periods, such as, buy and hold the stocks in the portfolios for 1 year or alternatively for 2 years
    11. Testing the system using different back-testing periods, such as from 1992 to 1999, from 2000 to 2007 and from 2008 to 2015

       

More info

To learn more about Alpha Captech's investment system please e-mail H. Mathiesen at hm@alphacaptech.com.

  

 

Important legal information: The contents located on this web site do not constitute an offer for sale. Alpha Captech does not make any promises that our past performance will guarantee similar future performance. Opinions expressed are subject to change at any time. The contents are based upon information which is considered reliable, but no representation is made that they are accurate or complete, and should not be relied upon.

Copyright 2018 Alpha Captech. All rights reserved.

Contact Alpha Captech