In mid-2013 TradingVisions released new system versions that incorporate a "walk-forward optimization" protocol, as well as new Vista WFO Portfolios that utilize these versions.
But what is walk-forward optimization?
Historically, the typical way to achieve strategies that are robust and give a reasonable expectation of future success is to test all available data for a market, adopt the "best" rules and parameters, and trade the system in real-time. We can call this approach look-backward optimization.
An improved approach is to not just look at all available data for one market, but to also look at additional markets, and to adopt the rules and parameters that best fit all or most of them. This approach--what I'll call look-beside optimization--is one I have used since I began developing systems over a dozen years ago.
I have been a firm believer that over-curve-fitting is the bane of system development, and to avoid it, I have been very reluctant to re-optimize systems once released. It is just too easy to find something that works on a single market for a few years in backtesting and re-jigger it every time results are poor.
My approach has worked quite well, but in early 2007 I began intensively testing a way of judiciously using re-optimization on Delphi II that proved to be preferable. This involves the use of walk-forward optimization (WFO), and it helps to solve two critical shortcomings of traditional optimization: 1.) traditional hypothetical performance records are idealized (optimized), "in-sample" results (i.e. results that are generated by the same data used to formulate the system rules & parameters), and these results usually vary widely from real-time results, and 2.) traditional optimization has no articulated way to adapt to different markets or to changes within a market.
But what if we had a way to find out how a system performs without waiting for years and risking our money? WFO optimizes a system over a set time period from the past--say three years--and applies the "best" parameters to a subsequent time period, say one year. The performance results of the three years study period are "in-sample," as defined above; in other words, the in-sample data was used to find the optimal values. These results are curve-fitted, idealized performance that is rarely matched in actual trading. In contrast, the performance on that one year after the study period is "out-of-sample," meaning that they are the result of trades that occur outside the period from which the rules and parameters were chosen. This is also called the "application period." The performance from the application period is much closer to what would have been achieved in actual trading, since the parameters were chosen prior to the trading, and we could theoretically have been trading during that time with that set of rules and parameters. In fact, if we use realistic slippage and commissions, we have what can be called "real-time" results, even though they are hypothetical in the sense that the trades were not actually made. These results are categorically more trustworthy than backtested results from a system's in-sample study period, and this is the most important advantage of WFO.
The "walk-forward" aspect of this approach comes in the next step. Let's say I decide that 3 years of data is sufficient to establish the rules and parameters of a system. If I used data from 1.1.2001 to 12.31.2003 as the first study period of the analysis, I would then determine my system rules and the ideal parameters from that 3 years. I could then run the system on 2004 data as my first application year, and the 2004 trading results are now my first year of the out-of-sample performance record. I now walk-forward a year and use the data from 1.1.2002 to 12.31.04 as my next study period, and I apply the "best" rules & parameters from this period to 2005. These 2005 out-of-sample results are added to my 2004 out-of-sample performance record. I continue to walk-forward to 1.1.2003-12.31.05, apply the best rules & parameters to 2006, and I now have a 3-year hypothetical out-of-sample track record that should be reasonably close to what I could have achieved in actual trading. This methodology is calling "rolling" or "running" walk-forward, as opposed to "anchored" walk-forward. The anchored approach does not drop the earlier "study" periods of data, which has the effect of making each study period increasingly longer. This method, which is the one used by TradingVisions, usually leads to increasing stabilization of the parameter choices (more on this below).
There are several additional benefits to WFO. In the past, I have worked very hard to have little or no difference in rules/parameters for different markets trading the same system. Optimizing to each market
using only in-sample data is very risky because there is no logical point at which to stop doing it. The almost inevitable result is over-curve-fitting and trading losses.
WFO allows for using different parameters for different markets because the backtested out-of-sample performance results for each market provide the validation of the methodology. When only in-sample data is used, the validation is based upon a circular relationship between the study data and the rules/parameters adopted: the results justify the rules/parameters, but the rules/parameters have been justified by the results. WFO takes us out of this pernicious loop and puts our expectations on a realistic foundation.