JEFFREY OWEN KATZ, Ph.D.
DONNA 1. M C CORMICK
TRADEMARKS AND SERVICE MARKS
Company and product names associated with listings in this book should be con-
sidered as trademarks or service marks of the company indicated. The use of a reg-
istered trademark is not permitted for commercial purposes without the permission
of the company named. In some cases, products of one company are offered by
other companies and are presented in a number of different listings in this book. It
is virtually impossible to identify every trademark or service mark for every prod-
uct and every use, but we would like to highlight the following:
Visual Basic, Visual C++, and Excel are trademarks of Microsoft Corp.
NAG function library is a service mark of Numerical Algorithms Group, Ltd.
Numerical Recipes in C (book and software) is a service mark of Numerical
TradeStation, SuperCharts, and SystemWriter Plus are trademarks of
Evolver is a trademark of Palisade Corporation.
Master Chartist is a trademark of Robert Slade, Inc.
TS-Evolve and TradeCycles (MESA) are trademarks of Ruggiero
Divergengine is a service mark of Ruggiero Associates.
C++ Builder, Delphi, and Borland Database Engine are trademarks
CQC for Windows is a trademark of CQG, Inc.
Metastock is a trademark of Eqnis International.
technical analysis function library is a service mark of FM Labs.
Excalibur is a trademark of Futures Truth.
MATLAB is a trademark of The MathWorks, Inc.
MESA96 is a trademark of Mesa.
What Is a Complete Mechanical Trading System? - What Are Good Entries and Exits?
* The Scientific Approach to System Development * Tools and Materials Needed for
the Scientific Approach
Tools of the Trade
Types of Data * Data Time Frames * Data Quality Data Sources and Vendors
Types of Simulators * Programming the Simulator * Simulator Output @erformance
summnry reports; trade-by-trade reports) * Simulator Perfomxmce (speed: capacity:
power) l Reliability of Simulators - Choosing the Right Simulator * Simulators Used
in This Book
Optimizers and Optimization 29
What Optimizers Do * How Optimizers Are Used * ?Lpes of Optimization (implicit
optimizers; brute force optimizers; user-guided optimization; genetic optimizers; optimization
by simulated annealing; analytic optimizers; linearpmgrwnming) How to Fail with
Optimization (small samples: large fxmztneter sets; no veri˜cation) . How to Succeed
with O&mization (h-ge, representative samples; few rules andparameters; veriicatim
@results) * Alternatives to Traditional Optimization * Optimizer Tools and Information *
Which Optimizer Is forYou?
Why Use Statistics to Evaluate Trading Systems? l Sampling * Optimization and
. Evaluating a System Statistically
Curve-Fitting l Sample Size and Representativeness
* Example 1: Evaluating the Out-of-Sample Test (what ifthe distribution is not normal?
what if there is serial dependence? what if the markets change?) l Example 2:
Evaluating the In-Sample Tests * Interpreting the Example Statistics (optimization
i-esults; verification results) l Other Statistical Techniques and Their Use (genetically
evoJved systems; multiple regression; monte car10 simulations; out-of-sample testing;
walk-forward testing) * Conclusion
The Study of Entries
What Constitutes a Good Entry? * Orders Used in Entries (stop orders; limit orders;
market orders; selecting appropriate orders) * Entry Techniques Covered in This Book
(breakouts and moving averages; oscillators; seasonality: lunar and solar phenomena:
cycles and rhythms; neural networks; geneticaNy evolved entry rules) * Standardized
Exits * Equalization of Dollar Volatility * Basic Test Portfolio and Platfcnm
Breakout Models 83
Kinds of Breakouts l Characteristics of Breakouts . Testing Breakout Models l
Channel Breakout Entries (close only channel breakouts; highest higMowest low
bnxzkouts) l Volatility Breakout Entries l Volatility Breakout Variations (long positions
only; currencies only; adx tremififilter) . Summary Analyses (breakout types: entry
orders; interactions; restrictions andjilters; analysis by market) * Conclusion l
What Have We Lamed?
Moving Average Models 109
What is a Moving Average? - Purpose of a Moving Average * The Issue of Lag l
Types of Moving Averages l Types of Moving Average Entry Models l Characteristics
of Moving Average Entries l Orders Used to Effect Entries * Test Methodology â€™
Tests of Trend-Following Models * Tests of Counter-Trend Models * Conclusion l
What Have We Learned?
Oscillator-Based Entries 133
What Is an Oscillator? l Kinds of Oscillators * Generating Entries with Oscillators *
Characteristics of Oscillator Entries . Test Methodology l Test Results (teas of
overbought/oversold models; tests of signal line models; tests of divergence models;
summary analyses) - Conclusion * What Have We Learned?
What Is Seasonality? l Generating Seasonal Entries l Characteristics of Seasonal
Entries . Orders Used to Effect Seasonal Entries . Test Methodology . Test Results
(test of the basic crossover model; tests of the basic momentum model: tests of the
crossover model with con$mtion; tests of the C˜SSOV˜˜ model with confirmation and
inversions: summary analyses) * Conclusion * What Have We Learned?
Lunar and Solar Rhythms 179
Legitimacy or Lunacy? l Lunar Cycles and Trading (generating lunar entries: lunar test
methodology; lunar test results; tests of the basic cmmo˜er model; tests of the basic
momentum model: tests of the cnx˜mer model with confirmation; test.s of the crmmver
model with confirmation and inversions; summary analyses; conclusion) * Solar
Activity and Trading (generazing solar entries: solar test results: conclusion) *
What Have We Learned?
Cycle-Based Entries 2Q3
Cycle Detection Using MESA l Detecting Cycles Using Filter Banks (butterworth
jilters; wavelet-basedjilters) * Generating Cycle Entries Using Filter Banks *
Characteristics of Cycle-Based Entries . Test Methodology . Test Results .
Conclusion l What Have We Learned?
Neural Networks 227
What Are Neural Networks? (feed-forward neural networks) . Neural Networks
in Trading l Forecasting with Neural Networks l Generating Entries with Neural
Predictions . Reverse Slow %K Model (code for the reverse slow % k model: test
methodology for the reverse slow % k model; training results for the reverse slow %k
model) l Turning Point Models (code for the turning point models; test methodology
for the turning point models; training resulrs for the turning point models) * Trading
Results for All Models (@ading results for the reverse slow %k model: frading results
for the bottom ruming point model; trading results for the top turning poinf model) *
Summary Analyses l Conclusion * What Have We Learned?
Genetic Algorithms 257
What Are Genetic Algorithms? * Evolving Rule-Based Entry Models * Evolving an
Entry Model @he rule remplares) * Test Methodology (code for evolving an entry
model) l Test Results (solutions evolved for long entries; solutions evolved for short
enrries; fesf results for the standard portfolio; market-by-market tesf resulrs: equify
curves; the rules for rhe solurions tesred) * Conclusion * What Have We Learned?
The Study of Exits
The Importance of the Exit l Goals of a Good Exit Strategy * Kinds of Exits
Employed in an Exit Strategy (money management exits; trailing exits; projir tnrgef
exiW rime-based exits; volarilify airs: barrier exits; signal exits) * Considerations
When Exiting the Market (gunning; trade-offs with prorecrive stops: slippage;
conC?nian rrading: conclusion) * Testing Exit Strategies * Standard Entries for Testing
Exits (the random entry model)
The Standard Exit Strategy 293
What is the Standard Exit Strategy? * Characteristics of the Standard Exit * Purpose of
Testing the SES l Tests of the Original SES (test results) * Tests of the Modified SES
(test resulrs) * Conclusion - What Have We Learned?
Improvements on the Standard Exit 309
Purpose of the Tests l Tests of the Fixed Stop and Profit Target * Tests of Dynamic
Stops (rest of the highest higWlowest low stop; fesf of the dynamic arr-based stop: fat
of the modified exponential moving average dynamic stop) * Tests of the Profit Taget *
Test of the Extended Time Limit - Market-By-Market Results for the Best Exit *
Conclusion l What Have We Learned?
Adding Artificial Intelligence to Exits 335
Test Methodology for the Neural Exit Component . Results of the Neural Exit Test
(baseline results; neural exit portjolio results: neural exit market-by-market results) .
Test Methodology for the Genetic Exit Component (top 10 solutions with baseline exit:
results of rule-based exits for longs and shorts; market-by-market resu1t.s (If rule-based
exits for longs: market-by-market results
I n this book is the knowledge needed to become a mc˜re successful trader of com-
modities. As a comprehensive reference and system developerâ€™s guide, the book
explains many popular techniques and puts them to the test, and explores innova-
tive ways to take profits out of the market and to gain an extra edge. As well, the
book provides better methods for controlling risk, and gives insight into which
methods perform poorly and could devastate capital. Even the basics are covered:
information on how to acquire and screen data, how to properly back-test systems
using trading simulators, how to safely perform optimization, how to estimate and
compensate for curve-fitting, and even how to assess the results using inferential
statistics. This book demonstrates why the surest way to success in trading is
through use of a good, mechanized trading system.
For all but a few traders, system trading yields mm-e profitable results than
discretionary trading. Discretionary trading involves subjective decisions that fre-
quently become emotional and lead to losses. Affect, uncertainty, greed, and fear
easily displace reason and knowledge as the driving forces behind the trades.
Moreover, it is hard to test and verify a discretionary trading model. System-
based trading, in contrast, is objective. Emotions are out of the picture. Through
programmed logic and assumptions, mechanized systems express the traderâ€™s
reason and knowledge. Best of all, such systems are easily tested: Bad systems
can be rejected or modified, and good cntes can be improved. This book contains
solid information that can be of great help when designing, building, and testing
a profitable mechanical trading system. While the emphasis is on an in-depth,
critical analysis of the various factors purported to contribute to winning systems,
the essential elements of a complete, mechanical trading system are also dissected
To be complete, all mechanical trading systems must have an entry method
and an exit method. The entry method must detect opportunities to enter the mar-
ket at points that are likely to yield trades with a good risk-to-reward ratio. The
exit method must protect against excessive loss of capital when a trade goes wrong
or when the market turns, as well as effectively capture profits when the market
moves favorably. A considerable amount of space is devoted to the systematic
back-testing and evaluation of exit systems, methods, and strategies. Even the
trader who already has a trading strategy or system that provides acceptable exits
is likely to discover something that can be used to improve the system, increase
profits, and reduce risk exposure.
Also included in these pages are trading simulations on entire pqrtfolios of
tradables. As is demonstrated, running analyses on portfolios is straightforward, if
not easy to accomplish. The ease of computing equity growth curves, maximum
drawdowns, risk-to-reward ratios, returns on accounts, numbers of trades, and all
the other related kinds of information useful in assessing a trading system on a
whole portfolio of commodities or stocks at once is made evident. The process of
conducting portfolio-wide walk-forward and other forms of testing and optimiza-
tion is also described. For example, instruction is provided on how to search for a
set of parameters that, when plugged into a system used to trade each of a set of
commodities, yields the best total net profit with the lowest drawdown (or perhaps
the best Sharpe Ratio, or any other measure of portfolio performance desired) for
that entire set of commodities. Small institutional traders (CTAs) wishing to run a
system on multiple tradables, as a means of diversification, risk reduction, and liq-
uidity enhancement, should find this discussion especially useful.
Finally, to keep all aspects of the systems and components being tested
objective and completely mechanical, we have drawn upon our academic and sci-
entific research backgrounds to apply the scientific method to the study of entry
and exit techniques. In addition, when appropriate, statistics are used to assess
the significance of the results of the investigations. This approach should provide the
most rigorous information possible about what constitutes a valid and useful com-
ponent in a successful trading strategy.
So that everyone will benefit from the investigations, the exact logic behind
every entry or exit strategy is discussed in detail. For those wishing to replicate
and expand the studies contained herein, extensive source code is also provided in
the text, as well as on a CD-ROM (see offer at back of book).
Since a basic trading system is always composed of two components, this
book naturally includes the following two parts: â€śThe Study of Entriesâ€ť and â€śThe
Study of Exits.â€ť Discussions of particular technologies that may be used in gener-
ating entries or exits, e.g., neural networks, are handled within the context of devel-
oping particular entry or exit strategies. The â€śIntroductionâ€ť contains lessons on the
fundamental issues surrounding the implementation of the scientific approach to
trading system development. The first part of this book, â€śTools of the Trade,â€ť con-
tains basic information, necessary for all system traders. The â€śConclusionâ€ť pro-
vides a summary of the research findings, with suggestions on how to best apply
the knowledge and for future research. The â€˜Appendixâ€ť contains references and
Finally, we would like to point out that this book is a continuation and elab-
oration of a series of articles we published as Contributing Writers to Technical
Analysis of Stocks and Commodities from 1996, onward.
Jeffrey Owen Katz, Ph.D., and Donna L. McCormick
There is one thing that most traders have in common: They have taken on the
challenge of forecasting and trading the financial markets, of searching for those
small islands of lucrative inefficiency in a vast sea of efficient market behavior.
For one of the authors, Jeffrey Katz, this challenge was initially a means to indulge
an obsession with mathematics. Over a decade ago, he developed a model that pro-
vided entry signals for the Standard & Poorâ€™s 500 (S&P 500) and OEX. While
these signals were, at that time, about 80% accurate, Katz found himself second-
guessing them. Moreover, he had to rely on his own subjective determinations of
such critical factors as what kind of order to use for entry, when to exit, and where
to place stops. These determinations, the essence of discretionary trading, were
often driven more by the emotions of fear and avarice than by reason and knowl-
edge. As a result, he churned and vacillated, made bad decisions, and lost more
often than won. For Katz, like for most traders, discretionary trading did not work.
If discretionary trading did not work, then what did? Perhaps system trading
was the answer. Katz decided to develop a completely automated trading system
in the form of a computer program that could generate buy, sell, stop, and other
necessary orders without human judgment or intervention. A good mechanical
system, logic suggested, would avoid the problems associated with discretionary
trading, if the discipline to follow it could be mustered. Such a system would pro-
vide explicit and well-defined entries, â€śnormalâ€ť or profitable exits, and â€śabnor-
malâ€ť or money management exits designed to control losses on bad trades,
A fully automated system would also make it possible to conduct historical
tests, unbiased by hindsight, and to do such tests on large quantities of data.
Thorough testing was the only way to determine whether a system really worked
and would be profitable to trade, Katz reasoned. Due to familiarity with the data
series, valid tests could not be performed by eye. If Katz looked at a chart and
â€śbelievedâ€ť a given formation signaled a good place to enter the market, he could
not trust that belief because he had already seen what happened after the forma-
tion occurred. Moreover, if charts of previous years were examined to find other
examples of the formation, attempts to identify the pattern by â€śeyeballingâ€ť would
be biased. On the other hand, if the pattern to be tested could be formally defined
and explicitly coded, the computer could then objectively do all the work: It
would run the code on many years of historical data, look for the specified for-
mation, and evaluate (without hindsight) the behavior of the market after each
instance. In this way, the computer could indicate whether he was indeed correct in
his hypothesis that a given formation was a profitable one. Exit rules could also
be evaluated objectively.
Finally, a well-defined mechanical trading system would allow such things
as commissions, slippage, impossible tills, and markets that moved before he
could to be factored in. This would help avoid unpleasant shocks when moving
from computer simulations to real-world trading. One of the problems Katz had in
his earlier trading attempt was failing to consider the high transaction costs
involved in trading OEX options. Through complete mechanization, he could
ensure that the system tests would include all such factors. In this way, potential
surprises could be eliminated, and a very realistic assessment could be obtained of
how any system or system element would perform. System trading might, he
thought, be the key to greater success in the markets.
WHAT IS A COMPLETE, MECHANICAL TRADING SYSTEM?
One of the problems witb Katzâ€™s early trading was that his â€śsystemâ€ť only provided
entry signals, leaving the determination of exits to subjective judgment; it was not,
therefore, a complete, mechanical trading system. A complete, mechanical trading
system, one that can be tested and deployed in a totally objective fashion, without
requiring human judgment, must provide both entries and exits. To be truly com-
plete, a mechanical system must explicitly provide the following information:
1. When and how, and possibly at what price, to enter the market
2. When and how, and possibly at what price, to exit the market with a loss
3. When and how, and possibly at what price, to exit the market with
The entry signals of a mechanical trading system can be as simple as explic-
it orders to buy or sell at the next dayâ€™s open. The orders might be slightly more
elaborate, e.g., to enter tomorrow (or on the next bar) using either a limit or stop.
Then again, very complex contingent orders, which are executed during certain
periods only if specified conditions are met, may be required-for example, orders
to buy or sell the market on a stop if the market gaps up or down more than so
many points at the open.
A trading systemâ€™s exits may also be implemented using any of a range of
orders, from the simple to the complex. Exiting a bad trade at a loss is frequently
achieved using a money management stop, which tertninates the trade that has
gone wrong before the loss becomes seriously damaging. A money management
stop, which is simply a stop order employed to prevent runaway losses, performs
one of the functions that must be achieved in some manner by a systemâ€™s exit strat-
egy; the function is that of risk control. Exiting on a profit may be accomplished
in any of several different ways, including by the use of pm@ targets, which are
simply limit orders placed in such a way that they end the trade once the market
moves a certain amount in the traderâ€™s favor; trailing stops, which are stop orders
used to exit with a profit when the market begins to reverse direction; and a wide
variety of other orders or combinations of orders.
In Katzâ€™s early trading attempts, the only signals available were of probable
direction or turning points. These signals were responded to by placing buy-at-
market or sell-at-market orders, orders that are often associated with poor fills and
lots of slippage. Although the signals were often accurate, not every turning point
was caught. Therefore, Katz could not simply reverse his position at each signal.
Separate exits were necessary. The software Katz was using only served as a par-
tially mechanical entry model; i.e., it did not provide exit signals. As such, it was
not a complete mechanical trading system that provided both entries and exits.
Since there were no mechanically generated exit signals, all exits had to be deter-
mined subjectively, which was one of the factors responsible for his trading prob-
lems at that time. Another factor that contributed to his lack of success was the
inability to properly assess, in a rigorous and objective manner, the behavior of the
trading regime over a sufficiently long period of historical data. He had been fly-
ing blind! Without having a complete system, that is, exits as well as entries, not
to mention good system-testing software, how could such things as net profitabil-
ity, maximum drawdown, or the Sharpe Ratio be estimated, the historical equity
curve be studied, and other important characteristics of the system (such as the
likelihood of its being profitable in the future) be investigated? To do these things,
it became clear-a system was needed that completed the full circle, providing
complete â€śround-turns,â€ť each consisting of an entry followed by an exit.
WHAT ARE GOOD ENTRIES AND EXITS?
Given a mechanical trading system that contains an entry model to generate entry
orders and an exit model to generate exit orders (including those required for
money management), how are the entries and exits evaluated to determine whether
they are good? In other words, what constitutes a good entry or exit?
Notice we used the terms entry orders and exit orders, not entry or exit sig-
nals. Why? Because â€śsignalsâ€ť are too ambiguous. Does a buy â€śsignalâ€ť mean that
one should buy at the open of the next bar, or buy using a stop or limit order? And
if so, at what price? In response to a â€śsignalâ€ť to exit a long position, does the exit
occur at the close, on a profit target, or perhaps on a money management stop?
Each of these orders will have different consequences in terms of the results
achieved. To determine whether an entry or exit method works, it must produce
more than mere signals; it must, at smne point, issue highly specific entry and exit
orders. A fully specified entry or exit order may easily be tested to determine its
quality or effectiveness.
In a broad sense, a good entry order is one that causes the trader to enter the
market at a point where there is relatively low risk and a fairly high degree of
potential reward. A traderâ€™s Nirvana would be a system that generated entry orders
to buy or sell on a limit at the most extreme price of every turning point. Even if
the exits were only merely acceptable, none of the trades would have more than
one or two ticks of adverse excursion (the largest unrealized loss to occur within
a trade), and in every case, the market would be entered at the best obtainable
price. In an imperfect world, however, entries will never be that good, but they can
be such that, when accompanied by reasonable effective exits, adverse excursion
is kept to acceptable levels and satisfying risk-reward ratios are obtained.
What constitutes an elective exit? An effective exit must quickly extricate the
trader from the market when a trade has gone wrong. It is essential to preserve cap-
ital from excessive erosion by losing trades; an exit must achieve this, however,
without cutting too many potentially profitable trades short by converting them into
small losses. A superior exit should be able to hold a trade for as long as it takes to
capture a significant chunk of any large move; i.e., it should be capable of riding a
sizable move to its conclusion. However, riding a sizable move to conclusion is not
a critical issue if the exit strategy is combined with an entry formula that allows for
reentry into sustained trends and other substantial market movements.
In reality, it is almost impossible, and certainly unwise, to discuss entries
and exits independently. To back-test a trading system, both entries and exits
must be present so that complete round-turns will occur. If the market is entered,
but never exited, how can any completed trades to evaluate be obtained? An entry
method and an exit method are required before a testable system can exist.
However, it would be very useful to study a variety of entry strategies and make
some assessment regarding how each performs independent of the exits.
Likewise, it would be advantageous to examine exits, testing different tech-
niques, without having to deal with entries as well. In general, it is best to manip-
ulate a minimum number of entities at a time, and measure the effects of those
manipulations, while either ignoring or holding everything else constant. Is this
not the very essence of the scientific, experimental method that has achieved so
much in other fields? But how can such isolation and control be achieved, allow-
ing entries and exits to be separately, and scientifically, studied?
THE SCIENTIFIC APPROACH TO SYSTEM DEVELOPMENT
This book is intended to accomplish a systematic and detailed analysis of the
individual components that make up a complete trading system. We are propos-
ing nothing less than a scientific study of entries, exits, and other trading system
elements. The basic substance of the scientific approach as applied herein is
1. The object of study, in this case a trading system (or one or more of its
elements), must be either directly or indirectly observable, preferably
without dependence on subjective judgment, something easily achieved
with proper testing and simulation software when working with com-
plete mechanical trading systems.
2. An orderly means for assessing the behavior of the object of study must
be available, which, in the case of trading systems, is back-testing over
long periods of historical data, together with, if appropriate, the applica-
tion of various models of statistical inference, the aim of the latter being
to provide a fix or reckoning of how likely a system is to hold up in the
future and on different samples of data.
3. A method for making the investigative task tractable by holding most
parameters and system components fixed while focusing upon the effects
of manipulating only one or two critical elements at a time.
The structure of this book reflects the scientific approach in many ways.
Trading systems are dissected into entry and exit models. Standardized methods for
exploring these components independently are discussed and implemented, leading
to separate sections on entries and exits. Objective tests and simulations are run,
and statistical analyses are performed. Results are presented in a consistent manner
that permits direct comparison. This is â€śold hatâ€ť to any practicing scientist.
Many traders might be surprised to discover that they, like practicing scien-
tists, have a working knowledge of the scientific method, albeit in different guise!
Books for traders often discuss â€śpaper tradingâ€ť or historical back-testing, or pre-
sent results based on these techniques. However, this book is going to be more
consistent and rigorous in its application of the scientific approach to the prob-
lem of how to successfully trade the markets. For instance, few books in which
historical tests of trading systems appear offer statistical analyses to assess valid-
ity and to estimate the likelihood of future profits. In contrast, this book includes
a detailed tutorial on the application of inferential statistics to the evaluation
of trading system performance.
Similarly, few pundits test their entries and exits independently of one
another. There are some neat tricks that allow specific system components to be
tested in isolation. One such trick is to have a set of standard entry and exit strate-
gies that remain fixed as the particular entry, exit, or other element under study is
varied. For example, when studying entry models, a standardized exit strategy will
be repeatedly employed, without change, as a variety of entry models are tested
and tweaked. Likewise, for the study of exits, a standardized entry technique will
be employed. The rather shocking entry technique involves the use of a random
number generator to generate random long and short entries into various markets!
Most traders would panic at the idea of trading a system with entries based on the
fall of the die; nevertheless, such entries are excellent in making a harsh test for
an exit strategy. An exit strategy that can pull profits out of randomly entered
trades is worth knowing about and can, amazingly, be readily achieved, at least for
the S&P 500 (Katz and McCormick, March 1998, April 1998). The tests will be
done in a way that allows meaningful comparisons to be made between different
entry and exit methods.
To summarize, the core elements of the scientific approach are:
1. The isolation of system elements
2. The use of standardized tests that allow valid comparisons
3. The statistical assessment of results
TOOLS AND MATERIALS NEEDED FOR THE SCIENTIFIC
Before applying the scientific approach to the study of the markets, a number of
things must be considered. First, a universe of reliable market data on which to
perform back-testing and statistical analyses must be available. Since this book is
focused on commodities trading, the market data used as the basis for our universe
on an end-of-day time frame will be a subset of the diverse set of markets supplied
by Pinnacle Data Corporation: these include the agriculturals, metals, energy
resources, bonds, currencies, and market indices. Intraday time-frame trading is
not addressed in this book, although it is one of our primary areas of interest that
may be pursued in a subsequent volume. In addition to standard pricing data,
explorations into the effects of various exogenous factors on the markets some-
times require unusual data. For example, data on sunspot activity (solar radiation
may influence a number of markets, especially agricultural ones) was obtained
from the Royal Observatory of Belgium.
Not only is a universe of data needed, but it is necessary to simulate one or
more trading accounts to perform back-testing. Such a task requires the use of a
trading simulator, a software package that allows simulated trading accounts to be
created and manipulated on a computer. The C+ + Trading Simulator from
Scientific Consultant Services is the one used most extensively in this book
because it was designed to handle portfolio simulations and is familiar to the
authors. Other programs, like Omega Researchâ€™s TradeStation or SystemWriter
Plus, also offer basic trading simulation and system testing, as well as assorted
charting capabilities. To satisfy the broadest range of readership, we occasionally
employ these products, and even Microsoftâ€™s Excel spreadsheet, in our analyses.
Another important consideration is the optimization of model parameters.
When running tests, it is often necessary to adjust the parameters of some compo-
nent (e.g., an entry model, an exit model, or some piece thereof) to discover the
best set of parameters and/or to see how the behavior of the model changes as its
parameters change. Several kinds of model parameter optimizations may be con-
ducted. In manual optimization, the user of the simulator specifies a parameter that
is to be manipulated and the range through which that parameter is to be stepped;
the user may wish to simultaneously manipulate two or more parameters in this
manner, generating output in the form of a table that shows how the parameters
interact to affect the outcome. Another method is brute force optimization, which
comes in several varieties: The most common form is stepping every parameter
through every possible value. If there are many parameters, each having many pos-
sible values, running this kind of optimization may take years. Brute force opti-
mization can, however, be a workable approach if the number of parameters, and
values through which they must be stepped, is small. Other forms of brute force
optimization are not as complete, or as likely to find the global optimum, but can
be run much more quickly. Finally, for heavy-duty optimization (and, if naively
applied, truly impressive curve-fitting) there are genetic algorithms. An appropri-
ate genetic algorithm (GA) can quickly tind a good solution, if not a global opti-
mum, even when large numbers of parameters are involved, each having large
numbers of values through which it must be stepped. A genetic optimizer is an
important tool in the arsenal of any trading system developer, but it must be used
cautiously, with an ever-present eye to the danger of curve-fitting. In the inves-
tigations presented in this book, the statistical assessment techniques, out-of-
sample tests, and such other aspects of the analyses as the focus on entire portfolios
provide protection against the curve-fitting demon, regardless of the optimization
Jeffrey Owen Katz, Ph.D., and Donna E McCormick
Tools of the Trade
T o obJectlve1y study the behavior of mechanical trading systems, various exper-
imental materials and certain tools are needed.
To study the behavior of a given entry or exit method, a simulation should be
done using that method on a portion of a given marketâ€™s past performance; that
requires data. Clean, historical data for the market on which a method is being
tested is the starting point.
Once the data is available, software is needed to simulate a trading account.
Such software should allow various kinds of trading orders to be posted and
should emulate the behavior of trading a real account over the historical period of
interest. Software of this kind is called a trading simulator.
The model (whether an entry model, an exit model, or a complete system)
may have a number of parameters that have to be adjusted to obtain the best results
from the system and its elements, or a number of features to be tamed on or off.
Here is where an optimizer plays its part, and a choice must be made among the
several types of optimizers available.
The simulations and optimizations will produce a plethora of results. The sys-
tem may have taken hundreds or thousands of trades, each with its own profiVloss,
maximum adverse excursion, and maximum favorable excursion. Also generated
will be simulated equity curves, risk-to-reward ratios, profit factors, and other infor-
mation provided by the trading simulator about the simulated trading account(s). A
way to assess the significance of these results is needed. Is the apparent profitabili-
ty of the trades a result of excessive optimization? Could the system have been prof-
itable due to chance alone, or might it really be a valid trading strategy? If the system
is valid, is it likely to hold up as well in the future, when actually being traded, as in
the past? Questions such as these require the basic machinery provided by inferen-
In the next several chapters, we will cover data, simulators, optimizers, and
statistics. These items will be used throughout this book when examining entry
and exit methods and when attempting to integrate entries and exits into complete
A determination of what works, and what does not, cannot be made in the realm
of commodities trading without quality data for use in tests and simulations.
Several types of data may be needed by the trader interested in developing a prof-
itable commodities trading system. At the very least, the trader will require his-
torical pricing data for the commodities of interest.
TYPES OF DATA
Commodities pricing data is available for individual or continuous contracts.
Individual contract data consists of quotations for individual commodities con-
tracts. At any given time, there may be several contracts actively trading. Most
speculators trade thefront-month contracts, those that are most liquid and closest
to expiration, but are not yet past first notice date. As each contract nears expira-
tion, or passes first notice date, the trader â€śrolls overâ€ť any open position into the
next contract. Working with individual contracts, therefore, can add a great deal of
complexity to simulations and tests. Not only must trades directly generated by the
trading system be dealt with, but the system developer must also correctly handle
rollovers and the selection of appropriate contracts.
To make system testing easier and more practical, the continuous contract was
invented. A continuous contract consists of appropriate individual contracts strung
together, end to end, to form a single, continuous data series. Some data massaging
usually takes place when putting together a continuous contract; the purpose is to
close the gaps that occur at rollover, when one contract ends and another begins,
Simple back-aajustment appears to be the most reasonable and popular gap-closing
method (Schwager, 1992). Back-adjustment involves nothing more than the sub-
traction of constants, chosen to close the gaps, from all contracts in a series other
than the most recent. Since the only operation performed on a contractâ€™s prices is the
subtraction of a constant, all linear price relationships (e.g., price changes over time,
volatility levels, and ranges) are preserved. Account simulations performed using
back-adjusted continuous contracts yield results that need correction only for
rollover costs. Once corrected for rollover, simulated trades will produce profits and
losses identical to those derived from simulations performed using individual con-
tracts. However, if trading decisions depend upon information involving absolute
levels, percentages, or ratios of prices, then additional data series (beyond back-
adjusted continuous contracts) will be required before tests can be conducted.
End-of-day pricing data, whether in the form of individual or continuous
contracts, consists of a series of daily quotations. Each quotation, â€śbar,â€ť or data
point typically contains seven fields of information: date, open, high, low,
close, volume, and open interest. Volume and open interest are normally
unavailable until after the close of the following day; when testing trading
methods, use only past values of these two variables or the outcome may be a
fabulous, but essentially untradable, system! The open, high, low, and close
(sometimes referred to as the settlement price) are available each day shortly
after the market closes.
Intraday pricing data consists either of a series of fixed-interval bars or of
individual ticks. The data fields for fixed-interval bars are date, time, open, high,
low, close, and tick volume. Tick volume differs from the volume reported for end-
of-day data series: For intraday data, it is the number of ticks that occur in the peri-
od making up the bar, regardless of the number of contracts involved in the
transactions reflected in those ticks. Only date, time, and price information are
reported for individual ticks: volume is not. Intraday tick data is easily converted
into data with fixed-interval bars using readily available software. Conversion soft-
ware is frequently provided by the data vendor at no extra cost to the consumer.
In addition to commodities pricing data, other kinds of data may be of value.
For instance, long-term historical data on sunspot activity, obtained from the
Royal Observatory of Belgium, is used in the chapter on lunar and solar influ-
ences. Temperature and rainfall data have a bearing on agricultural markets.
Various economic time series that cover every aspect of the economy, from infla-
tion to housing starts, may improve the odds of trading commodities successfully.
Do not forget to examine reports and measures that reflect sentiment, such as the
Commitment of Traders (COT) releases, bullish and bearish consensus surveys,
and put-call ratios. Nonquantitative forms of sentiment data, such as news head-
lines, may also be acquired and quantified for use in systematic tests. Nothing
should be ignored. Mining unusual data often uncovers interesting and profitable
discoveries. It is often the case that the more esoteric or arcane the data, and the
more difficult it is to obtain, the greater its value!
CMR I Data 5
DATA TIME FRAMES
Data may be used in its natural time frame or may need to be processed into a dif-
ferent time frame. Depending on the time frame being traded and on the nature of
the trading system, individual ticks, 5.minute bars, 20-minute bars, or daily, week-
ly, fortnightly (bimonthly), monthly, quarterly, or even yearly data may be neces-
sary. A data source usually has a natural time frame. For example, when collecting
intraday data, the natural time frame is the tick. The tick is an elastic time frame:
Sometimes ticks come fast and furious, other times sporadically with long inter-
vals between them. The day is the natural time frame for end-of-day pricing data.
For other kinds of data, the natural time frame may be bimonthly, as is the case for
the Commitment of Traders releases; or it may be quarterly, typical of company
Although going from longer to shorter time frames is impossible (resolution
that is not there cannot be created), conversions from shorter to longer can be read-
ily achieved with appropriate processing. For example, it is quite easy to create a
series consisting of l-minute bars from a series of ticks. The conversion is usual-
ly handled automatically by the simulation, testing, or charting software: by sim-
ple utility programs; or by special software provided by the data vendor. lf the data
was pulled from the Internet by way of ftp (tile transfer protocol), or using a stan-
dard web browser, it may be necessary to write a small program or script to con-
vert the downloaded data to the desired time frame, and then to save it in a format
acceptable to other software packages.
What time frame is the best? It all depends on the trader. For those attracted to
rapid feedback, plenty of action, tight stops, overnight security, and many small
profits, a short, intraday time frame is an ideal choice. On an intraday time frame,
many small trades can be taken during a typical day. The nttmeroous trades hasten the
learning process. It will not take the day trader long to discover what works, and
what does not, when trading on a short, intraday time frame. In addition, by closing
out all positions at the end of the trading day, a day trader can completely sidestep
overnight risk. Another desirable characteristic of a short time frame is that it often
permits the use of tight stops, which can keep the losses small on losing trades.
Finally, the statistically inclined will be enamored by the fact that representative data
samples containing hundreds of thousands of data points, and thousands of trades,
are readily obtained when working with a short time frame. Large data samples
lessen the dangers of curve-fitting, lead to more stable statistics, and increase the
likelihood that predictive models will perform in the future as they have in the past.
On the downside, the day trader working with a short time frame needs a real-
time data feed, historical tick data, fast hardware containing abundant memory, spe-
cialized software, and a substantial amount of time to commit to actually trading.
The need for fast hardware with plenty of memory arises for two reasons: (1)
System tests will involve incredibly large numbers of data points and trades; and
(2) the real-time software that collects the data, runs the system, and draws the
charts must keep up with a heavy flow of ticks without missing a beat. Both a data-
base of historical tick data and software able to handle sizable data sets are neces-
sary for system development and testing. A real-time feed is required for actual
trading. Although fast hardware and mammoth memory can now be purchased at
discount prices, adequate software does not come cheap. Historical tick data is like-
ly to be costly, and a real-time data feed entails a substantial and recurring expense.
In contrast, data costs and the commitment of time to trading are minimal for
those operating on an end-of-day (or longer) time frame. Free data is available on
the Internet to anyone willing to perform cleanup and formatting. Software costs
are also likely to be lower than for the day trader. The end-of-day trader needs less
time to actually trade: The system can be run after the markets close and trading
orders are communicated to the broker before the markets open in the morning:
perhaps a total of 15 minutes is spent on the whole process, leaving more time for
system development and leisure activities.
Another benefit of a longer time frame is the ability to easily diversify by simul-
taneously trading several markets. Because few markets offer the high levels of
volatility and liquidity required for day trading, and because there is a limit on how
many things a single individual can attend to at once, the day trader may only be able
to diversify across systems. The end-of-day trader, on the other hand, has a much
wider choice of markets to trade and can trade at a more relaxed pace, making diver-
sification across markets more practical than for intraday counterparts.
Diversification is a great way to reduce risk relative to reward. Longer time frame
trading has another desirable feature: the ability to capture large profits from strong,
sustained trends: these are the profits that can take a $50,000 account to over a mil-
lion in less than a year. Finally, the system developer working with longer time frames
will find more exogenous variables with potential predictive utility to explore.
A longer time frame, however, is not all bliss. The trader must accept delayed
feedback, tolerate wider stops, and be able to cope with overnight risk. Holding
overnight positions may even result in high levels of anxiety, perhaps full-blown
insomnia. Statistical issues can become significant for the system developer due to
the smaller sample sizes involved when working with daily, weekly, or monthly
data. One work-around for small sample size is to develop and test systems on
complete portfolios, rather than on individual commodities.
Which time frame is best? It all depends on you, the trader! Profitable trad-
ing can be done on many time frames. The hope is that this discussion has clar-
fied some of the issues and trade-offs involved in choosing correctly.
Data quality varies from excellent to awful. Since bad data can wreak havoc with
all forms of analysis, lead to misleading results, and waste precious time, only use
the best data that can be found when running tests and trading simulations. Some
forecasting models, including those based on neural networks, can be exceeding-
ly sensitive to a few errant data points; in such cases, the need for clean, error-free
data is extremely important. Time spent finding good data, and then giving it a
final scrubbing, is time well spent.
Data errors take many forms, some more innocuous than others. In real-time
trading, for example, ticks are occasionally received that have extremely deviant,
if not obviously impossible, prices. The S&P 500 may appear to be trading at
952.00 one moment and at 250.50 the next! Is this the ultimate market crash?
No-a few seconds later, another tick will come along, indicating the S&P 500 is
again trading at 952.00 or thereabouts. What happened? A bad tick, a â€śnoise
spike,â€ť occurred in the data. This kind of data error, if not detected and eliminat-
ed, can skew the results produced by almost any mechanical trading model.
Although anything but innocuous, such errors are obvious, are easy to detect (even
automatically), and are readily corrected or otherwise handled. More innocuous,
albeit less obvious and harder to find, are the common, small errors in the settling
price, and other numbers reported by the exchanges, that are frequently passed on
to the consumer by the data vendor. Better data vendors repeatedly check their
data and post corrections as such errors are detected. For example, on an almost
daily basis, Pinnacle Data posts error corrections that are handled automatically by
its software. Many of these common, small errors are not seriously damaging to
software-based trading simulations, but one never knows for sure.
Depending on the sensitivity of the trading or forecasting model being ana-
lyzed, and on such other factors as the availability of data-checking software, it
may be worthwhile to run miscellaneous statistical scans to highlight suspicious
data points. There are many ways to flag these data points, or ourlieru, as they are
sometimes referred to by statisticians. Missing, extra, and logically inconsistent
data points are also occasionally seen; they should be noted and corrected. As an
example of data checking, two data sets were run through a utility program that
scans for missing data points, outliers, and logical inconsistencies. The results
appear in Tables I-1 and 1-2, respectively.
Table I 1 shows the output produced by the data-checking program when it was
used on Pinnacle Data Corporationâ€™s (800-724-4903) end-of-day, continuous-con-
tract data for the S&P 500 futures. The utility found no illogical prices or volumes in
this data set; there were no observed instances of a high that wan less than the close,
a low that was greater than the open, a volume that was less than zero, or of any cog-
nate data faux pas. Rvo data points (bars) with suspiciously high ranges, however,
were noted by the software: One bar with unusual range occurred on 1 O/l 9/87 (or
871019 in the report). The other was dated 10/13/89. The abnormal range observed
on 10/19/87 does not reflect an error, just tbe normal volatility associated with a major
crash like that of Black Monday; nor is a data error responsible for the aberrant range
seen on 10/13/89, which appeared due to the so-called anniversary effect. Since these
statistically aberrant data points were not errors, corrections were unnecessary.
Nonetheless, the presence of such data points should emphasize the fact that market
events involving exceptional ranges do occur and must be managed adequately by a
trading system. All ranges shown in Table l-l are standardized ranges, computed by
dividing a barâ€™s range by the average range over the last 20 bars. As is common with
market data, the distribution of the standardized range had a longer tail than would be
Output from Data-Checking Utility for End-of-Day S&P 500
Continuous-Contract Futures Data from Pinnacle
CHAPTER 1 oata 9
expected given a normally distributed underlying process. Nevertheless, the events of
10/19/87 and 10/13/89 appear to be statistically exceptional: The distribution of all
other range data declined, in an orderly fashion, to zero at a standardized value of 7,
well below the range of 10 seen for the critical bars.
The data-checking utility also flagged 5 bars as having exceptionally deviant
closing prices. As with range, deviance has been defined in terms of a distribution,
using a standardized close-to-close price measure. In this instance, the standard-
ized measure was computed by dividing the absolute value of the difference
between each closing price and its predecessor by the average of the preceding 20
such absolute values. When the 5 flagged (and most deviant) bars were omitted,
the same distributional behavior that characterized the range was observed: a long-
tailed distribution of close-to-close price change that fell off, in an orderly fasb-
ion, to zero at 7 standardized units. Standardized close-to-close deviance scores
(DEV) of 8 were noted for 3 of the aberrant bars, and scores of 10 were observed
for the remaining 2 bars. Examination of the flagged data points again suggests
that unusual market activity, rather than data error, was responsible for their sta-
tistical deviance. It is not surprising that the 2 most deviant data points were the
same ones noted earlier for their abnormally high range. Finally, the data-check-
ing software did not find any missing bars, bars falling on weekends, or bars with
duplicate or out-of-order dates. The only outliers detected appear to be the result
of bizarre market conditions, not cormpted data. Overall, the S&P 500 data series
appears to be squeaky-clean. This was expected: In our experience, Pinnacle Data
Corporation (the source of the data) supplies data of very high quality.
As an example of how bad data quality can get, and the kinds of errors that
can be expected when dealing with low-quality data, another data set was ana-
lyzed with the same data-checking utility. This data, obtained from an acquain-
tance, was for Apple Computer (AAPL). The data-checking results appear in
In this data set, unlike in the previous one, 2 bars were flagged for having
outright logical inconsistencies. One logically invalid data point had an opening
price of zero, which was also lower than the low, while the other bar had a high
price that was lower than the closing price. Another data point was detected as
having an excessive range, which may or may not be a data error, In addition, sev-
eral bars evidenced extreme closing price deviance, perhaps reflecting uncorrect-
ed stock splits. There were no duplicate or out-of-order dates, but quite a few data
points were missing. In this instance, the missing data points were holidays and,
therefore, only reflect differences in data handling: for a variety of reasons, we
usually fill holidays with data from previous bars. Considering that the data series
extended only from l/2/97 through 1 l/6/98 (in contrast to the S&P 500, which ran
from l/3/83 to 5/21/98), it is distressing that several serious errors, including log-
ical violations, were detected by a rather simple scan.
The implication of this exercise is that data should be purchased only from a
Output from Data-Checking Utility for Apple Computer, Symbol AAPL
reputable vendor who takes data quality seriously; this will save time and ensure
reliable, error-free data for system development, testing, and trading, In addition,
all data should be scanned for errors to avoid disturbing surprises. For an in-depth
discussion of data quality, which includes coverage of how data is produced, trans-
mitted, received, and stored, see Jurik (1999).
DATA SOURCES AND VENDORS
Today there are a great many sowces from which data may be acquired. Data may
be purchased from value-added vendors, downloaded from any of several
exchanges, and extracted from a wide variety of databases accessible over the
Internet and on compact discs.
Value-added vendors, such as Tick Data and Pinnacle, whose data have been
used extensively in this work, can supply the trader with relatively clean data in
easy-to-use form. They also provide convenient update services and, at least in the
case of Pinnacle, error corrections that are handled automatically by the down-
loading software, which makes the task of maintaining a reliable, up-to-date data-
base very straightforward. Popular suppliers of end-of-day commodities data
include Pinnacle Data Corporation (800-724-4903), Prophet Financial Systems
(650-322-4183). Commodities Systems Incorporated (CSI, 800.274.4727), and
Technical Tools (800-231-8005). Intraday historical data, which are needed for
testing short time frame systems, may be purchased from Tick Data (SOO-822-
8425) and Genesis Financial Data Services (800-62 l-2628). Day traders should
also look into Data Transmission Network (DTN, SOO-485-4000), Data
Broadcasting Corporation (DBC, 800.367.4670), Bonneville Market Information
(BMI, 800-532-3400), and FutureSource-Bridge (X00-621 -2628); these data dis-
tributors can provide the fast, real-time data feeds necessary for successful day
trading. For additional information on data sources, consult Marder (1999). For a
comparative review of end-of-day data, see Knight (1999).
Data need not always be acquired from a commercial vendor. Sometimes it
can be obtained directly from the originator. For instance, various exchanges occa-
sionally furnish data directly to the public. Options data can currently be down-
loaded over the Internet from the Chicago Board of Trade (CBOT). When a new
contract is introduced and the exchange wants to encourage traders, it will often
release a kit containing data and other information of interest. Sometimes this is
the only way to acquire certain kinds of data cheaply and easily.
FiiaIly, a vast, mind-boggling array of databases may be accessed using an
Internet web browser or ftp client. These days almost everything is on-line. For exam-
ple, the Federal Reserve maintains files containing all kinds of economic time series
and business cycle indicators. NASA is a great source for solar and astronomical data.
Climate and geophysical data may be downloaded from the National Climatic Data
Center (NCDC) and the National Geophysical Data Center (NGDC), respectively.
For the ardent net-surfer, there is an overwh&lming abundance of data in a staggering
variety of formats. Therein, however, lies another problem: A certain level of skill is
required in the art of the search, as is perhaps some basic programming or scripting
experience, as well as the time and effort to find, tidy up, and reformat the data. Since
â€śtime is money,â€ť it is generally best to rely on a reputable, value-added data vendor
for basic pricing data, and to employ the Internet and other sources for data that is
more specialized or difficult to acquire.
Additional sources of data also include databases available through libraries
and on compact discs. ProQuest and other periodical databases offer full text
retrieval capabilities and can frequently be found at the public library. Bring a
floppy disk along and copy any data of interest. Finally, do not forget newspapers
such as Investorâ€™s Business Daily, Barronâ€™s, and the Wall Street Journal; these can
be excellent sources for certain kinds of information and are available on micro-
film from many libraries.
In general, it is best to maintain data in a standard text-based (ASCII) for-
mat. Such a format has the virtue of being simple, portable across most operating
systems and hardware platforms, and easily read by all types of software, from text
editors to charting packages.
No savvy trader would trade a system with a real account and risk real money
without first observing its behavior on paper. A trading simulator is a software
application or component that allows the user to simulate, using historical data, a
trading account that is traded with a user-specified set of trading rules. The userâ€™s
trading rules are written into a small program that automates a rigorous â€śpaper-
tradingâ€ť process on a substantial amount of historical data. In this way, the trad-
ing simulator allows the trader to gain insight into how the system might perform
when traded in a real account. The r&on dâ€™&re of a trading simulator is that it
makes it possible to efficiently back-test, or paper-trade, a system to determine
whether the system works and, if so, how well.
TYPES OF SIMULATORS
There are two major forms of trading simulators. One form is the integrated, easy-
to-use software application that provides some basic historical analysis and simu-
lation along with data collection and charting. The other form is the specialized
software component or class library that can be incorporated into user-written
software to provide system testing and evaluation functionality. Software compo-
nents and class libraries offer open architecture, advanced features, and high lev-
els of performance, but require programming expertise and such additional
elements as graphics, report generation, and data management to be useful.
Integrated applications packages, although generally offering less powerful simu-
lation and testing capabilities, are much more accessible to the novice.
PROGRAMMING THE SIMULATOR
Regardless of whether an integrated or component-based simulator is employed,
the trading logic of the userâ€™s system must be programmed into it using some com-
puter language. The language used may be either a generic programming lan-
guage, such as C+ + or FORTRAN, or a proprietary scripting language. Without
the aid of a formal language, it would be impossible to express a systemâ€™s trading
rules with the precision required for an accurate simulation. The need for pro-
gramming of some kind should not be looked upon as a necessary evil.
Programming can actually benefit the trader by encouraging au explicit and disci-
plined expression of trading ideas.
For an example of how trading logic is programmed into a simulator, consid-
er TradeStation, a popular integrated product from Omega Research that contains
an interpreter for a basic system writing language (called Easy Language) with bis-
torical simulation capabilities. Omegaâ€™s Easy Language is a proprietary, trading-
specific language based on Pascal (a generic programming language). What does a
simple trading system look like when programmed in Easy Language? The follow-
ing code implements a simple moving-average crossover system:
( Simple moving average crossover system in Easy Language)
(length parameter )
Inputs: kâ€ť(4) ;
rf (close > Average˜close, Led 1 And
(Close [II c= Average ˜ClOm?, Len) [II ) Then
myc â€˜Aâ€ť, 1 contract At Market; (buys at open of next bar)
If (Close <= Average (Close, Len), And
(Close L1, > *"verage (Close.
ILen) [II, Then
(sells at open Of next bar
Sellc"B"J 1 ContraCt At Market;
This system goes long one contract at tomorrowâ€™s open when the close crosses
above its moving average, and goes short one contract when the close crosses
below the moving average. Each order is given a name or identifier: A for the buy:
B for the sell. The length of the moving average (Len) may be set by the user or
optimized by the software.
Below is the same system programmed in Cf + using Scientific Consultant
Servicesâ€™ component-based C-Trader toolkit, which includes the C+ + Trading
Except for syntax and naming conventions, the differences between the Cf + and
Easy Language implementations are small. Most significant are the explicit refer-
ences to the current bar (cb) and to a particular simulated trading account or sim-
ulator class instance (ts) in the C+ + implementation. In C+ +, it is possible to
explicitly declare and reference any number of simulated accounts: this becomes
important when working with portfolios and merasystems (systems that trade the
accounts of other systems), and when developing models that incorporate an
implicit walk-forward adaptation.
All good trading simulators generate output containing a wealth of information
about the performance of the userâ€™s simulated account. Expect to obtain data on
gross and net profit, number of winning and losing trades, worst-case draw-
down, and related system characteristics, from even the most basic simulators.
Better simulators provide figures for maximum run-up, average favorable and
adverse excursion, inferential statistics, and more, not to mention highly detailed
analyses of individual trades. An extraordinary simulator might also include in
its output some measure of risk relative to reward, such as the annualized risk-
to-reward ratio (ARRR) or the Sharp Rario, an important and well-known mea-
sure used to compare the performances of different portfolios, systems, or funds
The output from a trading simulator is typically presented to the user in the
form of one or more reports. Two basic kinds of reports are available from most trad-
ing simulators: the performance summary and the trade-by-trade, or â€śdetail,â€ť report.
The information contained in these reports can help the trader evaluate a systemâ€™s
â€śtrading styleâ€ť and determine whether the system is worthy of real-money trading.
Other kinds of reports may also be generated, and the information from the
simulator may be formatted in a way that can easily be run into a spreadsheet for
further analysis. Almost all the tables and charts that appear in this book were pro-
duced in this manner: The output from the simulator was written to a file that
would be read by Excel, where the information was further processed and format-
ted for presentation.
Performance Summary Reports
As an illustration of the appearance of performance summary reports, two have
been prepared using the same moving-average crossover system employed to
illustrate simulator programming. Both the TradeStation (Table 2-l) and C-Trader
(Table 2-2) implementations of this system were run using their respective target
software applications. In each instance, the length parameter (controls the period
of the moving average) was set to 4.
Such style factors as the total number of trades, the number of winning trades, the
number of losing trades, the percentage of profitable trades, the maximum numbers
of consecutive winners and losers, and the average numbers of bars in winners and
losers also appear in performance summary reports. Reward, risk, and style are crit-
ical aspects of system performance that these reports address.
Although all address the issues of reward, risk and trading style, there are a
number of differences between various performance summary reports. Least sig-
nificant are differences in formatting. Some reports, in an effort to cram as much
information as possible into a limited amount of space, round dollar values to the
nearest whole integer, scale up certain values by some factor of 10 to avoid the
need for decimals, and arrange their output in a tabular, spreadsheet-like format.
Other reports use less cryptic descriptors, do not round dollar values or rescale
numbers, and format their output to resemble more traditional reports,
Somewhat more significant than differences in formatting are the variations
between performance summary reports that result from the definitions and
assumptions made in various calculations. For instance, the number of winning
trades may differ slightly between reports because of how winners are defined.
Some simulators count as a winner any trade in which the P/L (proWloss) figure
is greater than or equal to zero, whereas others count as winners only trades for
which the P/L is strictly greater than zero. This difference in calculation also
affects figures for the average winning trade and for the ratio of the average win-
ner to the average loser. Likewise, the average number of bars in a trade may be
greater or fewer, depending on how they are counted. Some simulators include the
entry bar in all bar counts; others do not. Return-on-account figures may also dif-
fer, depending, for instance, on whether or not they are annualized.
Differences in content between performance summary reports may even be
more significant. Some only break down their performance analyses into long
positions, short positions, and all trades combined. Others break them down into
in-sample and out-of-sample trades, as well. The additional breakdown makes it
easy to see whether a system optimized on one sample of data (the in-sample set)
shows similar behavior on another sample (the out-of-sample data) used for veri-
fication; out-of-sample tests are imperative for optimized systems. Other impor-
tant information, such as the total bar counts, maximum run-up (the converse of
drawdown), adverse and favorable excursion numbers, peak equity, lowest equity,
annualized return in dollars, trade variability (expressed as a standard deviation),
and the annualized risk-to-reward ratio (a variant of the Sharpe Ratio), are present
in some reports. The calculation of inferential statistics, such as the t-statistic and
its associated probability, either for a single test or corrected for multiple tests or
optimizations, is also a desirable feature. Statistical items, such as t-tests and prob-
abilities, are important since they help reveal whether a systemâ€™s performance
reflects the capture of a valid market inefficiency or is merely due to chance or
excessive curve-fitting. Many additional, possibly useful statistics can also be cnl-
culated, some of them on the basis of the information present in performance sum-
maries. Among these statistics (Stendahl, 1999) are net positive outliers, net neg-
ative outliers, select net profit (calculated after the removal of outlier trades), loss
ratio (greatest loss divided by net profit), run-up-t&rawdown ratio, longest flat
period, and buy-and-hold return (useful as a baseline). Finally, some reports also
contain a text-based plot of account equity as a function of time.
To the degree that history repeats itself, a clear image of the past seems like
an excellent foundation from which to envision a likely future. A good perfor-
mance summary provides a panoramic view of a trading methodâ€™s historical
behavior. Figures on return and risk show how well the system traded on test data
from the historical period under study. The Sharpe Ratio, or annualized risk to
reward, measures return on a risk- or stability-adjusted scale. T-tests and related
statistics may be used to determine whether a systemâ€™s performance derives from
some real market inefficiency or is an artifact of chance, multiple tests, or inap-
propriate optimization. Performance due to real market inefficiency may persist
for a time, while that due to artifact is unlikely to recur in the future. In short, a
good performance summary aids in capturing profitable market phenomena likely
to persist; the capture of persistent market inefficiency is, of course, the basis for
any sustained success as a trader.
This wraps up the discussion of one kind of report obtainable within most
trading simulation environments. Next we consider the other type of output that
most simulators provide: the trade-by-trade report.
Illustrative trade-by-trade reports were prepared using the simulators contained in
TradeStation (Table 2-3) and in the C-Trader toolkit (Table 2-4). Both reports per-
tain to the same simple moving-average crossover system used in various ways
throughout this discussion. Since hundreds of trades were taken by this system, the
original reports are quite lengthy. Consequently, large blocks of trades have been
edited out and ellipses inserted where the deletions were made. Because these
reports are presented merely for illustration, such deletions were considered
In contrast to a performance report, which provides an overall evaluation of
a trading systemâ€™s behavior, a detail or trade-by-trade report contains detailed
information on each trade taken in the simulated account. A minimal detail report
contains each tradeâ€™s entry and exit dates (and times, if the simulation involves
intraday data), the prices at which these entries and exits occurred, the positions
held (in numbers of contracts, long or short), and the profit or loss resulting from
each trade. A more comprehensive trade-by-trade report might also provide infor-
mation on the type of order responsible for each entry or exit (e.g., stop, limit, or
market), where in the bar the order was executed (at the open, the close, or in
Trade-by-Trade Report Generated by TradeStation for the Moving-
Average Crossover System
between), the number of bars each trade was held, the account equity at the start
of each trade, the maximum favorable and adverse excursions within each trade,
and the account equity on exit from each trade.
Most trade-by-trade reports contain the date (and time, if applicable) each
trade was entered, whether a buy or sell was involved (that is, a long or short posi-
tion established), the number of contracts in the transaction, the date the trade
was exited, the profit or loss on the trade, and the cumulative profit or loss on all
trades up to and including the trade under consideration. Reports also provide the
name of the order on which the trade was entered and the name of the exit order.
A better trade-by-trade report might include the fields for maximum favorable
excursion (the greatest unrealized profit to occur during each trade), the maxi-
mum adverse excursion (the largest unrealized loss), and the number of bars each
trade was held.
As with the performance summaries, there are differences between various
trade-by-trade reports with respect to the ways they are formatted and in the
assumptions underlying the computations on which they are based.
While the performance summary provides a picture of the whole forest, a good
trade-by-trade report focuses on the trees. In a good trade-by-trade report, each trade
is scrutinized in detail: What was the worst paper loss sustained in this trade? What
would the profit have been with a perfect exit? What was the actual profit (or loss)
Trade-by-Trade Report Generated Using the C-Trader Toolkit for the Moving-Average Crossover System
2 0 500
910527 -1 492.150 0 M 8: 910528 M A:
4550 6900 4550
910528 1 492.150 0 M A: 910607 M B: 11
-1 501.250 7 2975 4025 1525
0 B: 910613 0
910607 M M A:
1 495.300 0 M A: 910614 492.200 0
910613 M Et:
0 M A: 5 -2350 525 2825 3625
910614 -1 492.200 0 M 8: 910618 496.900
0 H B: 3 -2550 400 2550 1075
910618 1 496.900 0 M A: 910620 491.800
-1 491.800 490.650 H A: 6 575 1650 500
910620 0 M 9: 910625 1650
951225 1 692.600 0 M B: 8 550 1725 325 -15625
691.500 0 M A: 960101
7ca.700 0 M A: 4 -4050 1200 4050
960101 -1 692.600 0 M B: 960104 -19675
1 700.700 0 M B: 5 -4550 1675 5100 -24225
960104 0 M A: 960108 691.600
697.600 0 M A: 3 -3000 1450 3000 -27225
960108 -1 691.600 0 M B: 960110
0 0 moo -35525
M B: 2 -8300
0 M A: 960111 661.000
960110 1 697.600
683.000 0 M A: 8 -1000 4300 2325 -36525
960111 -1 6al.cQo 0 M B: 960118
1 663.000 729.300 0 M B: 30 23150 29050 1450 -I3375
96olltJ 0 M A: 960216
727.500 0 M A: 8 900 8400 1875 -l2475
960216 -1 729.300 0 M B:
960223 1 727.500 724.750 0 t-i B: 6-1375 5725 2750 -l3850
0 M A: 960228
960228 -1 724.750 722.900 0 M A: 7 925 Bl25 2525 -12925
0 N B: 960305
960308 725.900 0 M 8: 4l500 4475 1275 -11425
960305 1 722.900 0 H A:
960308 -1 725.900 716.150 0 M A: 4 4875 4075 1300 -6550
0 t4 E.: 960311
on the trade? Has the trading been fairly consistent? Are recent trades worse than
those of the past? Or are they better? How might some of the worst trades be char-
acterized in a way to improve the trading system? These are the kinds of questions
that cannOt be answered by a distant panoramic view of the forest (a summary
report), but they can be answered with a good trade-by-trade or detail report. In addi-
tion, a properly formatted detail report can be loaded into a spreadsheet for further
analysis. Spreadsheets are convenient for sorting and displaying data. They make it
easy, for instance, to draw histograms. Histograms can be very useful in decisions
regarding the placement of stops (Sweeney, 1993). Histograms can show how much
of the potential profit in the trades is being captured by the systemâ€™s exit strategy and
is also helpful in designing profit targets. Finally, a detailed examination of the worst
and best trades may generate ideas for improving the system under study.
Trading simulators vary dramatically in such aspects of performance as speed,
capacity, and power. Speed is important when there is a need to carry out many
tests or perform complex optimizations, genetic or otherwise. It is also essential
when developing systems on complete portfolios or using long, intraday data
series involving thousands of trades and hundreds of thousands of data points. In
some instances, speed may determine whether certain explorations can even be
attempted. Some problems are simply not practical to study unless the analyses
can be accomplished in a reasonable length of time. Simulator capacity involves
problem size restrictions regarding the number of bars on which a simulation may
be performed and the quantity of system code the simulator can handle. Finally,
the power a simulator gives the user to express and test complex trading ideas, and
to run tests and even system optimizations on complete portfolios, can be signifi-
cant to the serious, professional trader. A fairly powerful simulator is required, for
example, to run many of the trading models examined in this book.
The most significant determinant of simulation processing speed is the nature of
the scripting or programming language used by the simulator, that is, whether
the language is compiled or interpreted. Modern optimizing compilers for gener-
ic languages, such as Cf f, FORTRAN, and Pascal/Delphi, translate the user-
written source code into highly efficient machine code that the processor can
execute directly at full bore; this makes simulator toolkits that use such lan-
guages and compilers remarkably fast. On the other hand, proprietary, interpret-
ed languages, such as Microsoftâ€™s Visual Basic for Applications and Omegaâ€™s
Easy Language, must be translated and fed to the processor line by line.
Simulators that employ interpreted languages can be quite sluggish, especially
when executing complex or â€śloopyâ€ť source code. Just how much speed can be
gained using a compiled language over an interpreted one? We have heard
claims of systems running about 50 times faster since they were converted from
proprietary languages to C+ + !
While speed is primarily a function of language handling (interpreted versus com-
piled), capacity is mostly determined by whether 16-bit or 32-bit software is used.
Older, 16-bit software is often subject to the dreaded 64K limit. In practical terms,
this means that only about 15,000 bars of data (about 4 days of ticks, or 7 weeks of
l-minute bars on the S&P 500) can be loaded for system testing. In addition, as the
system code is embellished, expect to receive a message to the effect that the sys-
tem is too large to verify. Modem C+ + or FORTRAN products, on the other hand,
work with standard 32.bit C+ + or FORTRAN compilers. Consequently, they have
a much greater problem size capacity: With continuous-contract data on a machine
with sufficient memory, every single tick of the S&P 500 since its inception in 1983
can easily be loaded and studied! In addition, there are virtually no limits on the
number of trades a system can take, or on the systemâ€™s size and complexity. All
modern Cf +, FORTRAN, and Pascal/Delphi compilers are now full 32.bit pro-
grams that generate code for, and run under, 32-bit operating systems, such as
Windows 95, Windows NT, or LINUXAJNIX. Any simulator that works with such
a compiler should be able to handle large problems and enormous data sets with
ease. Since most software packages are upgrading to 32-bit status, the issue of
problem size capacity is rapidly becoming less significant than it once was.
Differences in simulator power are attributable mostly to language and to design.
Consider language first: In this case, it is not whether the language is compiled or
interpreted, as was the case for speed, but rather its expressive power. Can the most
elaborate and unusual trading ideas be expressed with precision and grace? In
some languages they can; in others they cannot. It is unfortunate that the most
powerful languages have steep learning curves. However, if one can climb the
curve, a language like Cf + makes it possible to do almost anything imaginable.
Your word processor, spreadsheet, web browser, and even operating system were
all probably written in C++ or its predecessor, C. Languages like C++ and
Object Pascal (the basis of Borlandâ€™s Delphi) are also extensible and can easily be
customized for the purpose of trading system development by the use of appro-
priate libraries and add-on components. Visual Basic and Easy Language,
although not as powerful as general-purpose, object-oriented languages like Cf +
or Object Pascal, have gentler learning curves and are still quite capable as lan-
guages go. Much less powerful, and not really adequate for the advanced system
developer, are the macro-like languages embedded in popular charting packages,
e.g., Bquis Internationalâ€™s MetaStock. The rule of thumb is the more powerful the
language, the more powerful the simulator.
Design issues are also a consideration in a simulatorâ€™s power. Extendability
and modularity are especially important. Simulators that employ C+ + or Object
Pascal (Borlandâ€™s Delphi) as their native language are incredibly extensible and
can be highly modular, because such general-purpose, object-oriented languages
are themselves highly extensible and modular; they were designed to be so from
the ground up. Class libraries permit the definition of new data types and opera-
tors. Components can provide encapsulated functionality, such as charting and
database management. Even old-fashioned function libraries (like the Numerical
Algorithms Group library, the International Mathematics and Statistics Library
and the Numerical Recipes library) are available to satisfy a variety of needs. Easy
Language, too, is highly extensible and modular: Modules called User Functions
can be created in Easy Language, and jimcrions written in other languages
(including C+ +) can be called (if they are placed in a dynamic link library, or
DLL). Macrolike languages, on the other hand, are not as flexible, greatly limit-
ing their usefulness to the advanced system developer. In our view, the ability to
access modules written in other languages is absolutely crucial: Different lan-
guages have different expressive foci, and even with a powerful language like
C+ +, it sometimes makes sense to write one or more modules in another lan
guage such as Prolog (a language designed for writing expert systems).
One additional design issue, unrelated to the language employed, is relevant
when discussing simulator power: whether a simulator can work with whole portfo-
lios as well as with individual tradables. Many products are not designed to perform
simulations and optimizations on whole portfolios at once, although sometimes add-
ons are available that make it possible to generate portfolio performance analyses
after the fact. On the other hand, an appropriately designed simulator can make mul-
tiple-account or portfolio simulations and system optimizations straightforward.
RELIABILITY OF SIMULATORS
Trading simulators vary in their reliability and trustworthiness. No complex soft-
ware, and that includes trading simulation software, is completely bug-free. This
is true even for reputable vendors with great products. Other problems pertain to
the assumptions made regarding ambiguous situations in which any of several
orders could be executed in any of several sequences during a bar. Some of these
items, e.g., the so-called bouncing tick (Ruggiero, 1998), can make it seem like the
best system ever had been discovered when, in fact, it could bankrupt any trader.
It seems better that a simulator makes worst-case assumptions in ambiguous situ-
ations: this way, when actual trading begins, there is greater likelihood of having
a pleasant, rather than an unpleasant, surprise. All of this boils down to the fact
that when choosing a simulator, select one that has been carefully debugged, that
has a proven track record of reliability, and in which the assumptions and handling
of ambiguous situations are explicitly stated. In addition, learn the simulatorâ€™s
quirks and how to work around them.
CHOOSING THE RIGHT SIMULATOR
If you are serious about developing sophisticated trading systems, need to work
with large portfolios, or wish to perform tests using individual contracts or
options, then buckle down, climb the learning curve, and go for an advanced sim-
ulator that employs a generic programming language such as the C++ or Object
Pascal. Such a simulator will have an open architecture that provides access to an
incredible selection of add-ons and libraries: technical analysis libraries, such as
those from PM Labs (609-261-7357) and Scientific Consultant Services (516-696-
3333); and general numerical algorithm libraries, such as Numerical Recipes
(800.872-7423), Numerical Algorithms Group (NAG) (44-1865.511.245), and
International Mathematics and Statistics Library (IMSL), which cover statistics,
linear algebra, spectral analysis, differential equations, and other mathematics.
Even neural network and genetic algorithm libraries are readily available.
Advanced simulators that employ generic programming languages also open up a
world of third-party components and graphical controls which cover everything
from sophisticated charting and data display to advanced database management,
and which are compatible with Borlandâ€™s C++ Builder and Delphi, as well as
with Microsoftâ€™s Visual Basic and Visual C+ +.
If your needs are somewhat less stringent, choose a complete, integrated solu-
tion Make sure the simulation language permits procedures residing in DLLs to be
called when necessary. Be wary of products that are primarily charting tools with
limited programming capabilities if your intention is to develop, back-test, and trade
mechanical systems that go significantly beyond traditional or â€ścannedâ€ť indicators.
SIMULATORS USED IN THIS BOOK
We personally prefer simulators built using modern, object-oriented programming
practices. One reason for our choice is that an object-oriented simulator makes it
easy to create as many simulation instances or simulated accounts as might be
desired. This is especially useful when simulating the behavior of a trading system
on an entire portfolio of tradables (as is done in most tests in this book), rather than
on a single instrument. An object-oriented simulator also comes in handy when
building adaptive, self-optimizing systems where it is sometimes necessary to
implement internal simulations. In addition, such software makes the construction
of merasysrems (systems that trade in or out of the equity curves of other systems)
a simple matter. Asset allocation models, for instance, may be treated as metasys-
terns that dynamically allocate capital to individual trading systems or accounts. A
good object-oriented simulator can generate the portfolio equity curves and other
information needed to create and back-test asset allocation models operating on
top of multiple trading systems. For these reasons, and such others as familiarity,
most tests carried out in this book have been performed using the C-Trader toolk-
it. Do not be alarmed. It is not necessary to have any expertise in C+ + or mod-
em software practices to benefit from this book. The logic of every system or
system element examined will be explained in great detail in the text.
Optimizers and Optimization
I t would be nice to develop trading systems without giving a thought to opti-
mization. Realistically, however, the development of a profitable trading strategy
is a trial-and-error activity in which some form of optimization always plays a
role. There is always an optimizer around somewhere-if not visible on the table,
then lurking in the shadows.
An @mizer is simply an entity or algorithm that attempts to find the best
possible solution to a problem; optimization is the search process by which that
solution is discovered. An optimizer may be a self-contained software component,
perhaps implemented as a C++ class, Delphi object, or ActiveX control.
Powerful, sophisticated optimizers often take the form of software components
designed to be integrated into user-written programs. Less sophisticated opti-
mizers, such as those found in high-end simulation and charting packages, are usu-
ally simple algorithms implemented with a few lines of programming code. Since
any entity or algorithm that performs optimization is an optimizer, optimizers need
not be associated with computers or machines at all; an optimizer may be a per-
son engaged in problem-solving activities. The human brain is one of the most
powerful heuristic optimizers on earth!
WHAT OPTIMIZERS DO
Optimizers exist to find the best possible solution to a problem. What is meant by
the best possible solution to a problem? Before attempting to define that phrase,
let us first consider what constitutes a solution. In trading, a solution is a particu-
lar set of trading rules and perhaps system parameters,
All trading systems have at least two mles (an entry rule and an exit rule), and
most have one or more parameters. Rules express the logic of the trading system,
and generally appear as â€śif-thenâ€ť clauses in whatever language the trading system has
been written. Parameters determine the behavior of the logic expressed in the rules;
they can include lengths of moving averages, connection weights in neural networks,
thresholds used in comparisons, values that determine placements for stops and prof-
it targets, and other similar items. The simple moving-average crossover system,
used in the previous chapter to illustrate various trading simulators, had two rules:
one for the buy order and one for the sell order. It also had a single parameter, the
length of the moving average. Rules and parameters completely define a trading sys-
tem and determine its performance. To obtain the best performance from a trading
system, parameters may need to be adjusted and rules juggled.
There is no doubt that some rule and parameter combinations define systems
that trade well, just as others specify systems that trade poorly; i.e., solutions dif-
fer in their quality. The goodness of a solution or trading model, in terms of how
well it performs when measured against some. standard, is often calledjfitness. The
converse of fitness, the inadequacy of a solution, is frequently referred to as coot.
In practice, fitness is evaluated by a@ness function, a block of programming
code that calculates a single number that reflects the relative desirability of any
solution. A fitness function can be written to appraise fitness howsoever the trader
desires. For example, fitness might be interpreted as net profit penalized for exces-
sive drawdown. A costfuncrion works in exactly the same way, but higher numbers
signify worse solutions. The sum of the squared errors, commonly computed when
working with linear regression or neural network models, is a cost function.
The best possible solution to a problem can now be defined: It is that partic-
ular solution that has the greatest fitness or the least cost. Optimizers endeavor to
find the best possible solution to a problem by maximizing fitness, as measured by
a titness function, or minimizing cost, as computed by a cost function.
The best possible solution to a problem may be discovered in any number of
ways. Sometimes problems can be solved by simple trial-and-error, especially when
guided by human insight into the problem being worked. Alternatively, sophisticated
procedures and algorithms may be necessary. For example, simulating the process of
evolution (as genetic optimizers do) is a very powerful way to discover or evolve
high-quality solutions to complex problems. In some cases, the best problem solver
is an analytic (calculus-based) procedure, such as a conjugate gradient. Analytic opti-
mization is an efficient approach for problems with smooth (differentiable) fitness
surfaces, such as those encountered in training neural networks, developing multiple
linear regression models, or computing simple-structure factor rotations.
HOW OPTIMIZERS ARE USED
Optimizers are wonderful tools that can be used in a myriad of ways. They help
shape the aircraft we fly, design the cars we drive, and even select delivery routes