Download (direct link):
System Development and Analysis
clude all the various types of market environments imaginable: bullish, bearish, sideways and trending.
The other factor instrumental in determining how much data will be sufficient is the typical duration of the systemâ€™s trades. The longer the duration, the more data required to ensure a statistically significant sampling. This is why I included 10 years of data history for the long- to intermediateterm trading systems but only 7 months of history for our 5-minute trading system.
Data history for the long- to intermediate-term trading systems in Chapters 3 and 4 ran from December 31, 1992, to December 31, 2002. I set aside the year 2003 for use in this chapter to ensure that these systems continue to work with the most recent data sampling available. (This practice is known as walk-forward or out-of-sample testing. If done correctly, it increases the probability of a high correlation between backtested performance results and those experienced in real-time trading accounts.) Why did I decide to use 10 years of backtesting on long- to intermediate-term systems with one year (2003) as my out-of-sample data history? Although I could have chosen to go back farther (e.g., 20 years), the farther we go back in time, the less our data will tend to reflect current market dynamics.
A comparison of the performance results from 1993 to 2002 (see Figure 7.1) of the channel breakout system for spot British pound-U.S. dollar versus the results generated from 1983 to 2002 in Figure 7.2 emphatically illustrates the point that more is not always better. Of course the data from 1983 to 1992 should be considered valid; but should it be given the same weighting as last yearâ€™s data? Probably not.
Perhaps a viable alternative is retention of the longer-term data with the introduction of an exponential or linear weighting factor. This solution enables retention of the larger data sampling while simultaneously giving greater weight to the most recent data history. Obviously employment of such a solution raises its own problems, namely how much weighting on the most recent data is too much and how much is too little. Unfortunately, as with the question of the length of the data series itself, there are no absolute answers. Instead, readers are encouraged to experiment with various weighting factors on a case-by-case basis until a reasonable and statistically significant solution is found.
System Integrity: Avoiding the Pitfalls
As stated at the beginning of this chapter, because the future will never look exactly the same as the past, our goal as system developers is to generate real-time performance results that display as high a positive correlation to backtested data as possible. We already have seen how assurance
FIGURE 7.1 Spot British pound/U.S. dollar with 20-day channel breakout. Includes data from December 30, 1992, to December 30, 2002.
Note: All trade summaries include $100 round-turn trade deductions for slippage and commissions. Â©2004 CQG, Inc. All rights reserved worldwide.
FIGURE 7.2 Spot British pound/U.S. dollar with 20-day channel breakout. Includes data from December 31, 1982, to December 31, 2002.
Note: All trade summaries include $100 round-turn trade deductions for slippage
and commissions. Â©2004 CQG, Inc. All rights reserved worldwide.
System Development and Analysis
of data integrity aids in attainment of this goal. Next we examine how the integrity of the trading system itself can ensure the highest possible correlations between our historical and future performance.
Checking the Systemâ€™s Integrity
The ability of computers to generate and backtest various concepts quickly and efficiently has been one of the great leaps forward for traders and system developers. Because we can now determine profitability, win/loss ratios, and profit to maximum drawdown with such ease, it is tempting simply to tinker through the virtually limitless permutations of trading parameters until we find those that best resonate with our trading personality types. This â€śtinkeringâ€ť process satisfies both the scientist and the trader in us, and our ability to generate backtested performance results with such speed and efficiency often lulls us into a false sense of security regarding accuracy. Consequently, I remind readers that generated performance results are only as accurate as the ability of programming code to capture desired entry and exit conditions.
The only way to avoid the problem of faulty programming code and to ensure the integrity of a trading systemâ€™s performance results is the painstaking and tedious process known as spot checking. Spot checking entails running through the entire trade list results for the system and comparing these against the full data history. Even prior to combing through the details of the entire data history, a quick review of the performance summary tables often can clue us in to programming flaws. Examples of these anomalies in performance tables include 0 or 100 percent of trading signals generated being long positions, average length of trades being atypically short or long in duration or 0 or 100 percent of trades being losses.