One of the most significant issues in system trading is "real" versus "ideal" performance numbers, so I'd like to provide a few informal definitions and discuss this important facet of automated trading, as well as provide some data.
Computers are powerful tools that allow us to analyze myriad data. When we state trading rules in analysis software like TradeStation and run tests on historical data, we are "backtesting." With computers we can run thousands of backtests, finding the needle of a sharp idea in a haystack of numbers. Anyone can look at a chart, spot a pattern consisting of a few examples, and see where they should have entered and exited, but the computer allows us to find hundreds of examples of the same conditions to see whether trading that pattern is consistent enough to be profitable over time. A computer further allows us to refine our ideas through "optimization," which is a test of a numerical idea, running through a series of values to find the optimal one(s). Backtested, optimized results are idealized because they give us the "best" fit to the curves of past data, or they are "curve-fitted." The data used as our testing period is called the sample data, and to distinguish it from other data, it is called "in-sample" data. Results that are from past data are also called "in-sample," meaning that they are derived from the tested data sample.
Unfortunately, although backtesting is a powerful tool, devising a system that shows profitable backtesting does not provide assurance of profits in a trading account. This is because markets are constantly evolving and tend to mutate. As we must say, "Past performance does not guarantee future results." We therefore should not place too much credence in backtested, in-sample, optimized, curve-fitted, idealized results. Instead, we look to results that come from "out-of-sample" data. This data is any that resides outside the original test data used. It can be from an earlier period of time, a later period of time, or from a different market (e.g. the e-mini Russell, instead of the e-mini S&P). The results from out-of-sample data have much more import because they tell us how our system trades in real markets, as opposed to the ideal one of our test sample. When those results are from data after the end of our study data, it is "real-time" results (some people use "real-time" to mean results from actual trades placed in an account, but I prefer to use it as any trades generated after the end period of in-sample data). The results are "real" in the sense that the trades could have been taken in real time, since the system already existed, and they were not results from the "ideal" time period over which the system was developed.
"Hypothetical" has two primary meanings in the context of trading. Less strictly it means results from the in-sample data set. More strictly it means any results that were not actual trades in an account. Unfortunately, this latter category masks an important distinction in hypothetical results. If the results are from the in-sample period, then they are idealized results. But if they are results from the out-of-sample period that may or may not have been actually traded in an account, then they have much more significance because they could have been real trades.
In judging a system, the real-time, out-of-sample, unoptimized results should carry more weight than the in-sample results, if there are a significant number of trades. We want to know how the system performed in less-than-ideal conditions.
The TradingVisions systems--particularly the WFO series--have a significant amount of out-of-sample, unoptimized performance, which in this case means performance that either occurred after the release of the system rules and parameters or performance based on out-of-sample data. If, for example, a system was released July 15, 2004, and it has not been substantively changed since then, the period after 7/15/04 is considered to be out-of-sample and real-time. Or if an optimization was run over the period 1/1/2000 to 12/31/2003, then any results from after that end date would be considered out-of-sample.
TradingVisions System Release Dates & Out-of-sample Periods
Futures trading systems and commodity trading bear a high degree of risk. People can and do lose money. Hypothetical results have many inherent limitations. Past performance does not guarantee future results. Please read thedisclosures & disclaimerspage.
HYPOTHETICAL PERFORMANCE RESULTS HAVE MANY INHERENT LIMITATIONS, SOME OF WHICH ARE DESCRIBED BELOW. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN. IN FACT THERE ARE FREQUENTLY SHARP DIFFERENCES BETWEEN HYPOTHETICAL PERFORMANCE RESULTS AND THE ACTUAL RESULTS SUBSEQUENTLY ACHIEVED BY ANY PARTICULAR TRADING PROGRAM. ONE OF THE LIMITATIONS OF HYPOTHETICAL PERFORMANCE RESULTS IS THAT THEY ARE GENERALLY PREPARED WITH THE BENEFIT OF HINDSIGHT. IN ADDITION, HYPOTHETICAL PERFORMANCE TRADING DOES NOT INVOLVE FINANCIAL RISK, AND NO HYPOTHETICAL TRADING RECORD CAN COMPLETELY ACCOUNT FOR THE IMPACT OF FINANCIAL RISK IN ACTUAL TRADING. FOR EXAMPLE, THE ABILITY TO WITHSTAND LOSSES OR TO ADHERE TO A PARTICULAR TRADING PROGRAM IN SPITE OF TRADING LOSSES ARE MATERIAL POINTS WHICH CAN ALSO ADVERSELY AFFECT ACTUAL TRADING RESULTS. THERE ARE NUMEROUS OTHER FACTORS RELATED TO THE MARKETS IN GENERAL OF THE IMPLEMENTATION OF ANY SPECIFIC TRADING PROGRAM WHICH CANNOT BE FULLY ACCOUNTED FOR IN THE PREPARATION OF HYPOTHETICAL PERFORMANCE RESULTS AND ALL OF WHICH CAN ADVERSELY AFFECT ACTUAL TRADING RESULTS.
No representation is being made that any account will achieve similar results to any actual account numbers or hypothetical numbers cited herein or at the TradingVisions site. Results can vary significantly from brokerage to brokerage, depending upon many factors not under the control of TradingVisions. The contents of this e-mail are provided for informational purposes only, and this is not an offer or solicitation to buy or sell any securities. While the information contained herein is believed to be reliable, we cannot guarantee its completeness or accuracy.
Unless otherwise stated, hypothetical results reported by TradingVisions are those generated by the latest version of the systems and recommended portfolios, including the specific rules and parameter settings. It is therefore not advisable, nor is it the intended purpose, to use these hypothetical results as a guide to what past results should have been achieved by utilizing the version of the systems or recommended portfolio in effect at a past time. In some cases, slightly different entry and exit times and prices may be used in leased and/or purchased copies of the trading systems, in an attempt to lessen the impact of multiple orders reaching the market at or close to the same time. While over time and an extended number of trades these differences have in the past tended to be small and immaterial, they may in fact prove to have a material impact or over shorter timeframes have a material impact. Portfolio recommendations may change without notice, and TradingVisions has neither the right nor responsibility to change what is traded in client accounts.
Neither TradeStation Technologies nor any of its affiliates has reviewed, certified, endorsed, approved, disapproved or recommended, and neither does or will review, certify, endorse, approve, disapprove or recommend, any trading software tool that is designed to be compatible with the TradeStation Open Platform.
TradingVisions Systems, Inc. | 351 Palisades Dr N, Sedona, AZ | 928-554-4052