<<

. 8
( 9)



>>

ting profits run or, at least, not cutting them short.
No combinations of parameters yielded profitable results. Given that this is
a minimalist exit strategy, and that random entries were being used, such an out-
come was expected. Nevertheless, the performance of the exit strategy across dif-
ferent parameter combinations can be compared.


Conclusion
Profits should not be cut short even though a higher percentage of winning trades
might be gained. Doing so may make the average trade a larger loser or smaller
winner. In addition, there appears to be an optima1 placement for a fixed money
management stop. Too wide a stop increases the percentage of wins. However, it
also increases the overall loss. Too tight a stop keeps the individual losses small,
but drastically cuts the percentage of winning trades, again resulting in worse
overall performance. An optimal value provides a moderate percentage of winning
trades and the best performance. In this case, the optimal distance to place the
money management stop away from the entry price was 1.5 average true range
units. With some entry systems, the optima1 placement might be much closer.


TESTS OF DYNAMIC STOPS
In this group of tests, the fixed money management stop of the MSES is replaced
with what will; it is hoped, be a much better stop. The placement of the stop is
dynamically adjusted. The goal is to capture as much unrealized profit as possible,
while not turning potentially profitable trades into losing ones, as might happen
with an overly tight, fixed stop.
There are many ways to adjust the stop so that it follows or trails the market,
locking in some of the profit that develops in the course of a trade. One popular
method is, for long positions, to place the stop for the next bar at the lowest of the
lows of the current and previous bars. The stop is only placed at progressively
higher levels, never lower levels. For short positions, the stop is placed at the high-
est of the current bar™s and previous bar™s highs, It is only allowed to move down,
never up. This simple methodology is examined in the first test.
The second test is of a dynamic stop that mimics the fixed stop used in the
MSES. The stop is adjusted up (long) or down (short) based on the current price
minus (long) or plus (short) a multiple of the average true range. Unlike the stop
used in the MSES, where the placement was fixed, the placement revisions of
this stop are based on current market value. The revised placements may only
occur in one direction: up for long positions, down for short ones. The intention
is to keep the stop a similar statistical distance from the best current price
achieved at each stage of the trade, just as it was from the original entry price.
The stop for long positions is calculated as follows: (1) Subtract from the entry
price a parameter (mmstp) multiplied by the average true range (the result is the
stop for the next bar). (2) On the next bar, subtract from the current price anoth-
er parameter (stpa) multiplied by the average true range. (3) If the stop resulting
from the calculations in Step 2 is greater than the current stop, then replace the
current stop with the resultant stop: otherwise the stop remains unchanged. (4)
Repeat Steps 2 and 3 for each successive bar. In the stop for short positions, the
multiples of the average true range are added to the market prices and the stops
are ratcheted down.
The third test involves a more sophisticated approach. For long positions, the
stop is initialized a certain number of average true range units below the entry
price, as usual. The stop is then moved up an amount that is determined by how
far the current prices are above the stop™s current value. For short positions, the ini-
tialization occurs above the entry price, and the stop is moved down to an extent
determined by how far the current prices are below it. The stop initially moves in
quickly, losing momentam as it reaches the vicinity of the current price level. This
method involves nothing more than a kind of offset exponential moving average
(EMA), except that the moving average is initialized in a special way on entry to
the trade and is only allowed to move in one direction; i.e., the stop is never polled
further away from the market, only closer. The stop for long positions is calculat-
ed as follows: (1) Initialize the stop to be used on the entry bar by subtracting from
the entry price a parameter (mmstp) multiplied by the average true range. (2) At
the next bar, subtract from the high a parameter (stpa) multiplied by the average
true range; then subtract the current placement of the stop, and finally multiply by
another parameter (srpb). (3) If the resultant number from Step 2 is greater than
zero, then add that number to the value of the current stop; otherwise the stop
remains unaltered. (4) Repeat Steps 2 and 3 for each successive bar. In the stop for
short positions, the multiples of the average true range are added to the market
prices and only corrections that are negative are added to the stop prices.

static void Model (float *parms, float Wt. float *opn, float *hi.
float *lo, float l cls, float *vol. float *oi, float ˜dlrv, int nb,
TRDSIM as, float *eqclsl {

// Implements random entry tests of the standard exit
// strategy modified to use “dynamic” stops.
,, File = x20mod02.c
vector [I. .MAxPFm, Of parameters
// pa-s
vector [I. .nbl of dates in YYMMDD form
// d-L
vector [I. .rlbl Of opening prices
// opn
318



v e c t o r [l. .*I o f h i g h
prices
vector Ll. .*I of low prices
- vector [l..nbl Of c1oaing prices
vector [I. .*I Of volumes
vector [i..nbl of open interest numbers
- vector [l..nh] of average dollar volatilities
number of bars in data series or vectors
trading simulator class instance
vector Il..*1 of closing equity levels

// declare local scratch variables
static int rc, cb. *contracts, maxhold, signal, ranseed;
static float stpa, stpb, matp, ptlim, limprice, stpprice;
static int entryposted, entrybar, modeltype;
static float exitatrM+XBAR+ll, mum, entryprice, tmp, at*;
static long iseed;

,, copy parameters to local variables for clearer reference
mmstp = parms[ll; // used to set the initial stop
= parms[2]; // additional stop parameter
stpa
stp!J = parms131; I/ additional stop parameter
ptlim = parma[61; I/ profit target limit in at* units
modeltype = parms˜71; // type of dynamic stop to use
= parmsL81; // maximum holding period in days
maxhold
= parms˜91 ; ,, used to Select random seed
ranseed

I/ perform whole-series calculations
˜˜gTruenange˜˜exitatr,hi,lo,cls.50,*˜; // ATR for exit

,, seed the random number generator
// _._ use a different seed for each tradeable
I/ t*.modelO returns a market index CSP=l, YX=2. .)
iwed = -(ranseed + 10 * ts.modelo,;
mum = rarOC&iseed);
break;
case 2:
stpprice = min(stpprice, cls[cbl+stpa*a˜r˜;
break;
case 3:
tmp = (lolcbl + stpa l atrl stpprice;

if˜mp < 0.0) stpprice += stptz * elnp;

break;


)

tS.exit6hortlimit(˜*˜, limprice˜;

ts .exitdlortstop (˜J™ , stpprice1 i

iftch-entrybar >= maxhold) ts.exitshortclose(˜H™);


1

I

) ,/ process next bar


I




The code above implements the MSES with the original fixed stop replaced
with one of the three more responsive, dynamic stops, The parameter modeltype
selects the stop, and depending on which is to be tested, up to three other con-
trolling parameters are set and adjusted. For the two-bar highest-high/lowest-
low (HHLL) stop, the parameter (mmsrp) is the multiple of the average true
range that is added to or subtracted from the entry price to obtain the stop price
for the entry bar. The stop price on the entry bar is set to either the lowest low
of the last two bars or the entry price plus or minus the specified multiple of the
average true range, whichever is further away from the current market price. For
the other two kinds of stops (ATR-based and MEMA), the stop price on the entry
bar is initialized to the usual value, i.e., the entry price minus (long) or plus
(short) the money management stop parameter (mmsrp) multiplied by the aver-
age true range.
On each bar after the entry bar, the stop price is adjusted. The adjustment
used depends on the particular type of stop being employed, as selected by the
modeltype parameter. For the HHLL stop, the highest high or lowest low is cal-
culated according to whether the position is short or long (respectively). If the
result of the calculation is closer to the current market price than the current stop
price is, then the current stop price is replaced with the new value. For the next
model (dynamic ATR-based stop), a second money management stop parameter
(srpa) is multiplied by the average true range. Then the resultant number is sub-
tracted from (long) or added to (short) the current closing price. If the resultant
number is closer to the market than the current value of the stop price, the stop
price is replaced with that number and, therefore, moved in closer to the market.
For the third type of stop (MEMA), a stop parameter (stpa) is multiplied by the
average true range, and then subtracted from the current high (longs) or added to
the current low (shorts) as a kind of offset. The stop price is then subtracted from
322




the resultant number. The result of this last calculation is placed in a variable
(nap). The stop price is updated on successive bars by adding tmp multiplied by
another parameter (sfpb, a correction rate coefficient) to the existing stop price.
The current stop price is adjusted, however, only if the adjustment will move the
stop closer to the current price level. The computations are identical to those used
when calculating an exponential moving average (EMA). The only difference is
that, in a standard exponential moving average, the stop price would be correct-
ed regardless of whether the correction was up or down and there would be no
offset involved. In this model, stpb determines the effective length of an expo-
nential moving average that can only move in one direction, in toward the prices.

Test of Highest-High/Lowest-Low Stop
In this test (modeltype = l), tire initial money management stop parameter con-
trols the maximum degree of the stop™s tightness on the first bar. It is stepped from
0.5 to 3.5 in increments of 0.5.
Each row in Table 14-2 presents data on the in-sample performance of each
of the values through which the parameter was stepped (ZNSTP). The last row
describes the behavior of the model with the best parameter value (as found in the
stepping process) retested on the out-of-sample data. Table 14-2 may be interpret-
ed in the same way as the other optimization tables presented in this book, in the
table, DRAW represents drawdown, in thousands of dollars.
This stop appears to have been consistently too tight, as evidenced by a decreased
percentage of winning trades when compared with the baseline MSFS model. In the
previous test, the best solution (mmcrp parameter 1.5, ptlim parameter 4.5) had 39%
wins, had a risk-to-reward ratio of - 1.46, and lost an average of $1,581 per trade. In
the current test, the best solution has only 28% of the trades winning in-sample and
29% out-of-sample. Many potentially profitable trades (some of the trades that would
have been profitable with the basic MSES, using an optimal fixed stop) were convert-
ed to small losses. The tightness of this stop is also demonstrated by the total number
of bats the average trade was held (4), compared with the usual 6 to 8 bars. The aver-
age risk-to-reward ratio (-2.52 in-sample and -2.38 out-of-sample) and the average
loss per trade ($1,864 in-sample, $1,818 out-of-sample) have also significantly wors-
ened when compared with the optimal fixed stop. The 2-bar HI-ILL stop is obviously
no great shakes, and one would be better served using a fixed, optimally placed stop,
like that which forms part of the MSES when used with optimized parameters, such
as those discovered and shown in the test results in Table 14-1.

Test of the Dynamic ATR-Based Stop
In this model, the two parameters (represented in the code as mmsip and apa) are
multipliers for the average true range. They are used when calculating placement of
the stop on the entry bar and later bars, respectively. The entry bar stop parameter
323




TABLE 14-P

Portfolio Performance for the l-Bar Highest-High/Lowest-Low Trailing
Stop with the Optimal Fixed Profit Target Found in Table 14-l




I I I I
OUT-OF-SAMPLE I I I I I
I
2.50 [ -32591 -15111 -1748l -22.41 -2.381 1.000˜ 17931 281 -1SlSl
0.711 34501 4




(FIRST in Table 14-3) and the parameter for the stop after the entry bar (LATER) are
both stepped from 0.5 to 3.5 in increments of 0.5.
As with the previous tests of stops, the parameters had a gradual effect on
performance and did not interact with one another to any great extent. An exami-
nation of the average performance for each value of the entry bar stop parameter
reveals that the best results, in terms of risk-to-reward ratio, were obtained when
that parameter was set to 2. For the parameter that applied to later bars, the best
average result was achieved with values of either 2 or 2.5. For individual combi-
nations of parameters, a FIRST parameter of 2 and a LATER parameter of 2.5 pro-
duced the best overall performance, with the least bad risk-to-reward ratio and
nearly the smallest loss per trade. This stop model was marginally better than the
optimal fixed stop, used as a baseline, that had a risk-to-reward ratio of - 1.46, as
opposed to - 1.40 in the current case. As in Table 14-1, the best solution is shown
in boldface type. The percentage of winning trades (42%) was also marginally bet-
ter than that for the optimal fixed stop (39%).

Test of the MEMA Dynamic Stop
There are three parameters in this model: the initial money management stop pxa-
meter, which sets the stop for the first bar; the ATR offset parameter (ATRO in
Table 14-4); and the correction or adaptation rate coefficient (COEFF), which
determines the relative rate at which the stop pulls in to the market or, equivalently,
the speed of the modi@ed exponential moving average underlying this model. All
three parameters are optimized with an extensive search.
Table 14-4 shows portfolio performance only as a function of the ATR offset
and the adaptation rate coefficient, the most important parameters of the model.
The initial stop parameter was fixed at 2.5, which was the parameter value of the
optimal solution.
32.4




Portfolio Performance as a Function of First-Bar and Later-
Bar Stop Parameters Using the Dynamic ATR-Based Stop-
Loss Model




Again, the model responded in a well-behaved manner to changes in the para-
meter values. There was some interaction between parameter values. This was
expected because the faster the moving average (or adaptation rate), the greater the
average true range offset must be to keep the stop a reasonable distance from the
prices and achieve good performance. The best overall portfolio performance (the
boldfaced results in Table 14-4) occurred with an ATR offset of 1 and an adaptation
rate coefficient of 0.3 (about a 5-bar EMA). Finally, here is a stop that performed
better than those tested previously. The risk-to-reward ratio rose (- 1.36), as did the
percentage of winning trades (37%) and the average trade in dollars ( - $1,407).

TEST OF THE PROFIT TARGET
This test is the best stop thus far produced: The MEMA stop had an initial money
management parameter of 2.5, an average true range offset of 1, and an adaptation
rate coefficient of 0.30. In the original MEMA test above (the results reported in
Table 14-4), the optimal fixed profit target was used. In the current test, the optimal
fixed profit target is replaced with a shn™nking profit target, i.e., one that starts out
far away from the market and then pulls in toward the market, becoming tighter
over time. The intention is to try to pull profit out of languishing trades by exiting
with a limit order on market noise, while not cutting profits short early in the course
of favorably disposed trades. The approach used when constructing the shrinking


TABLE 14-4

Portfolio Performance of the EMA-Like Dynamic Stop as a Function
of the ATR Offset and Adaptation Rate Coefficient with an Initial Stop
Parameter of 2.5 ATR Units
326



profit target is very similar to the one used when constructing the MEMA stop. An
exponential moving average is initialized in an unusual way; i.e., the running sum
is set to the entry price plus (long) or minus (short) some multiple @Tim) of the
average true range. In this way, the profit target limit begins just as the fixed profit
target limit began. After the first bar, the price at which the limit is set is adjusted
in exactly the same way that an exponential moving average is adjusted as new bars
arrive: The distance between the current limit price and the current close is multi-
plied by a parameter @rga). The resultant number is then subtracted from the cur-
rent limit price to give the new limit price, pulling the limit price in tighter to the
current close. In contrast to the case with the stop, the limit price is allowed to move
in either direction, although it is unlikely to do so because the limit order will take
the trade out whenever prices move to the other side of its current placement. The
second parameter (p&z) controls the speed of the moving average, i.e., the shrink
age rate. The rules are identical to those in the test of the MEMA stop above, except
as they relate to the profit target limit on bars after the first bar.




limprice = entryprice + ptlim l atr;

stpprice = min ˜Lawest˜lo, 2, cbl,

entryprice mstp + atr,;

break;

case 2:

case 3:

case 4:

limprice = entryprice + ptlim * atr;

stpprice = entryprice mmtp * atr;

break;
327



stpprice = max (Highestthi. 2, cb),
entryprice + mwtp * atr);
break;
case 2:
case 3:
case 4:
limprice = entryprice ptlim l atr;

stpprice = entryprice + nmletp f atr;

break;
default: nrerror("mvalid modeltype");
I
ts.exitshortlimit('C', limprice);
ts.exitshortstop('D', stpprice);
1
else (




stpprice = max(stpprice, Lowesf(lo,Z,cb));
break;
case 2:
stppcice = max(stpprice, cls[cbl -stpa*atr);
break;
case 3:
trap = tbiicbl atpa * at=) - stpprice;
if(tmp > 0.0) stpprice += stpb l tmp;
break;
case 4:
mp = ChiIcbl sepa * atrl sfpprice;
if(tmp > 0.0) stpprice += stpb * tmp;
limprice = limprice - ptga'(limprice-cls[cbl);
break;
)
ts.exiflonglimit('F', limprice);
ts.exitlongstopC'G', stpprice);
if(cb-entrybar >= maxhold) ts.exitlongclose('E'l;
1
else if(ts.positionO < 0) ( // shorts
ewitch(modeltype) (
case 1:
stpprice = min(stpprice, Highesf(hi,P,cb));
break;
case 2:
stpprice = min(stpprice, cls[cbl+stpa*atr);
break;
328



case 3:




ts.exitshortatop(˜J™, stpprice);

if<&-entrybar >= maxhold) ts.exitshortclose(˜H™);


1

)




The code fragment above shows the implementation of the shrinking limit,
along with the MEMA stop reported in Table 14-4.
Table 14-5 provides information on the portfolio performance as the initial
profit target limit @lim in the code, INIT in the table) and the shrinkage coeffi-
cient @&a in the code, COEFF in the table) are varied. The parameter that con-
trolled the initial placement of the profit target, in average tree range units away
from the price, was stepped from 2 to 6 in increments of 0.5. The shrinkage coef-
ficient was stepped from 0.05 to 0.4 in increments of 0.05. The best combination
of parameters produced a solution that was an improvement over the fixed profit
target limit. The risk-to-reward ratio became - 1.32, the percentage of winning
trades remained the same at 37%, but the average loss per trades was less at
$1,325. Again, the model was well behaved with respect to variations in the para-
meters. The results indicate that care has to be taken with profit targets: They tend
to prematurely close trades that have large profit potential. As can be seen in Table
14-5, as the initial profit target placement became tighter and tighter, the percent-
age of winning trades dramatically increased, as more and more trades hit the prof-
it target and were closed out with a small profit. On the down side, the
risk-to-reward ratio and the average worsened, demonstrating that the increased
percentage of winning trades could not compensate for the curtailment of profits
that resulted from the tight profit target. Sometimes it is better to have no profit
target at all than to have an excessively tight one. The same was tme for the shrink-
age rate. Profit targets that too quickly moved into the market tended to close
trades early, thereby cutting profits short.


TEST OF THE EXTENDED TIME LIMIT
In all the tests conducted so far, a position was held for only a maximum of 10
days. Any position that still existed-that had not previously been closed out by
Portfolio Performance as a Function of the initial Profit Target Limit
Placement and the Shrinkage Coefficient




the stop or profit target-was closed out after 10 days, regardless of its profitabil-
ity. In this test, an exit strategy that uses an adaptive MEMA stop, with optimal
parameters and a shrinking profit target, is examined. The only difference between
the test reported in Table 14-6 and the one reported in Table 14-5 is in the exten-
sion of the maximum time a trade may be held, from 10 to 30 days. The initial
profit target limit is reoptimized by stepping it from 5 to 7 in increments of 0.5.
Likewise, the shrinkage rate coefficient is stepped from 0.05 to 0.4 in increments
of 0.05. The code is the same as for the previous test. Only the setting of the max-
hold parameter, which controls the maximum time a trade may be held, is
changed.
The best performance was achieved with an initial target parameter of 5.5
and a shrinkage coefficient of 0.1. The average risk-to-reward ratio went from
- 1.32 to ˜ 1.22. The percentage of wins remained the same, but the average trade
lost only $1,236, rather than $1,325 in the previous test. Extension of the time
limit improved results, but not dramatically. Most trades were closed out well
before the time limit expired; i.e., the average trade only lasted between 6 and 10
bars (days).


MARKET-BY-MARKET RESULTS FOR THE BEST
EXIT
Table 14-7 reports on performance broken down by market for the best exit strat-
egy discovered in these tests, i.e., the one that used the MEMA stop and the
shrinking profit target and in which the time limit was extended to 30 days. Both
in- and out-of-sample results are presented, and consistency between the two can



Portfolio Performance as a Function of the Initial Profit Target Limit
Setting and the Shrinkage Coefficient When the Trade Limit Is
Increased to 30 Days
be seen.
In both samples, profits for the NYFE occurred on the long side, but not on
the short. Substantial profits occurred in both samples for Feeder Cattle-for long
and short positions in-sample, but only for short positions out-of-sample. Live
Hogs were profitable in both samples for both long and short positions. The
Deutschemark and Japanese Yen showed profits in-sample on the long side, but
had overall losses out-of-sample. The only exception was a small profit in the
Japanese Yen on the short side, but it was not enough to overcome the losses on
the long side. Lumber was strongly profitable on the long side in-sample, but only


TABLE 14-7

Performance Broken Down by Market for Best Exit Strategy
with EMA-Like Stop, Shrinking Profit Target, and Trade Time Limit
Extended to 30 Days
332 PART III The Study of Exits




had a very small profit on the long side out-of-sample. The two most outstanding
performers were Feeder Cattle and Live Hogs which, even despite the random
entries, could actually be traded. In-sample, there was a 10.9% annualized return
for Feeder Cattle and a 15.5% return for Live Hogs. Out-of-sample, the returns
were 43.1% and 3 1.9%, respectively. There were more profitable results out-of-
sample than in-sample, but this could easily have been due to the smaller sample
and fewer trades taken in the out-of-sample period.

CONCLUSION
Exits do make a big difference. By improving the risk management and profit tar-
get elements in an exit strategy, losses can be cot and the risk-to-reward ratio can
be enhanced. The improvements in the tests above, however, were not as good as
expected. For example, although the best exit did appear capable of pulling prof-
its from random trades taken in two markets, no profits were obtained on the port-
folio, which is somewhat inconsistent with our earlier experiences (Katz and
McCormick, March 1998, April 1998) in which profitable systems were achieved
with random entries on the S&P 500. In those studies, exits were toned to the mar-
ket under examination, rather than keeping parameters constant across an entire
portfolio as done in the investigations above. This difference may account for the
poorer results in the current set of tests. In general, better results can be obtained
(albeit with much greater risk of curve-fitting and over-optimization) by tuning the
various components of a trading model to the specific characteristics of an indi-
vidual market. It should also be kept in mind that the tests conducted here were
fairly harsh with respect to transaction costs. For some markets (e.g., the S&P
500), commissions are almost negligible in terms of its typical dollar volatility,
and only slippage is a factor. However, in many smaller markets, great numbers of
contracts would have to be traded, causing the issue of commissions to become a
very significant consideration. In our earlier study, little or no transaction costs
were assumed, and a market in which the commission component would be fairly
small (i.e., the S&P 500) was examined. This factor may also have contributed to
the difference in findings.
When compared with the standard exit strategy used in the tests of entry
methods, which lost an average of $2,243 per trade and had a standard deviation of
$304, the best exit strategy thus far developed reduced the loss per trade to $1,236,
representing a reduction in loss per trade of over 44%. The reduction is substantial
enough that many of the better (albeit losing) entry models would probably show
overall profitability if they were combined with the best exit strategy.
333




WHAT HAVE WE LEARNED?

. Exits can make a substantial difference in overall performance. The
attempts described in this chapter have yielded an extra $1,000 per trade
over the standard exit strategy used in tests of entry models.
. Just as with entries, finding a good exit is like searching for a tiny island
of inefficiency in a sea of &cient market behavior. While such islands
are there, they are difficult to find.
CHAPTER IS


ADDING Al TO EXITS




I n this chapter, the modified standard exit strategy (MSES) is explored with the
addition of elements based on neural networks and genetics. In Chapter 11, neur-
al network forecasters were developed for use in generating entries. One of the
particular neural forecasters (the time-reversed Slow %K net) attempted to predict
whether tomorrow™s prices would be near the low or high end of the range of
prices that would occur over the next several days. This network can be added to
an exit strategy: If the net suggests that the market is near the top of its near-future
range and a long position is being held, it would probably he a good idea to exit
before the market begins to move down. Likewise, if the net forecasts a rising
market while a short position is being held, the trade should be exited before the
market begins to rise.
The first test conducted below explores the use of the time-reversed Slow
%K network (developed when studying entries) as an additional element to our
modified standard exit strategy. The net, which generates what might be called a
signal exit, cannot be used on its own for exiting because it will not always close
out a trade. This network was designed to provide entry signals. When it generates
a signal, the market is likely to behave in some expected manner. However, the
absence of a signal does not mean that the market will not do something signifi-
cant. When a position is being held, at some point an exit has to be taken, and that
action cannot be postponed until a significant event is finally predicted. The
MSES, in this case, guarantees that all trades have some money management pro-
tection and are exited after a given amount of time. The neural net, however, can
possibly improve the strategy by occasionally triggering an exit when a predicted
move against the trade is expected. In this way, the net may turn a certain propor-
tion of losing trades into winners.
The second batch of tests (one for the long side, one for the short) involves the
use of a genetic algorithm to evolve a set of rules tcr generate a signal exit. The rules
are used in a manner similar to the way the net is used, i.e., to generate additional
exits, witbin the context of the MSES, when the market is likely to reverse. The rule
templates and rule-generating methodology are the same as those used in Chapter
12, where evolved rules generated entries. In the current tests, rules are evolved to
generate additional exits within the context of the MSES. The rules are used as sig-
nal exits. The additional exits will, it is hoped, improve profitability by turning some
losses into wins and by killing other trades before they become larger losses.
More sophisticated exits can be developed using the techniques described above.
Although not explored in this chapter, a neural network could be evolved to produce
outputs in the form of placements for stops and limits, as well as for immediate, out-
right exits. Genetically evolved rules could also be employed in this manner.
When used in the context of entries, neural networks proved to be fairly good
forecasters. It-sample, incredible profits were produced due to the accurate pre-
dictions. Out-of-sample, the nets yielded much better than chance results (albeit
not very profitable on the whole portfolio). Real predictive ability was demon-
strated. Using such forecasts to exit trades before the market reverses should
improve system performance, even if only by eliminating a small percentage of
bad trades. The same applies to the genetically evolved rules. However, when the
rules that produced entry signals for rare event trades are applied to exits, great
improvement in exit performance should not be expected. Given the nature of the
rules, only a small number of signals are likely to be generated, which means that
only small numbers of trades will be affected. If only a few of the large number of
trades that will be taken are improved, only a small overall benefit will be evi-
denced. Since the rules are being reevolved for the tests below, more inslances of
exit opportunities may be found than were discovered for entry opportunities.

TEST METHODOLOGY FOR THE NEURAL EXIT
COMPONENT
The larger of the two best neural networks, trained to predict the time-reversed
Slow %K, are used. The preprocessing and forecast-generating logic are identical
to those discussed in Chapter 11. A series of predictions are generated using the
18-14-4-1 net (18 first-layer neurons, 14 neurons in first middle layer, 4 in the sec-
ond middle layer, and 1 output). The MSES is also used. Along with the exits pro-
vided by the MSES, an additional exit condition is being added: If the predicted
reverse Slow %K is greater than some threshold, indicating that the market is high
relative to its near-future price range, any long position is exited. Likewise, if the
net™s prediction indicates that the market is near the low of its neat-future price
range, by being below 100 minus the previous threshold, any short position is exit-
ed. Exits triggered by the neural net forecasts are taken at the close of the bar.
338




// exit trades using the modified standard exit
// strategy along with the neural signal exit
iflentrypasted z 0) (
// initialization and exits for longs on entry day
limprice I entryprice + pt1im * exitatrLcb1;
stpprice = entryprice - mstp * exitatr[cbl ;
ts.exi˜longlimit(,A', limprice);
339



ts.exitlo*gstop˜˜B™, srppsice);
if(prd[cbl > thresh) ta.exitlongclose('C');
1
e1ae if˜entryposted < 0) (
/I initialization and exits for shorts on entry day
limprice - entryprice - ptlim * exitatr[cbl;
stpprice = entryprice + mmstp * exiratr[cbl;
ta.exitshortlimit('o', limprice);
t˜.exiCshortst˜p('E', atpprice);
if(prdLcbl < loo.o-thresh, te.exirshortclose('P'l:
)
else (




The code fragment above implements the logic of the exit strategy. The para-
meters ptlim and mmsrp are set to 4.5 and 1.5, respectively; these are the values
that gave the best overall portfolio performance (see Table 14-1, Chapter 14). The
thresh parameter, i.e., the threshold used to generate exits based on the neural fore-
casts, is optimized. The logic of the additional exit can be seen in the “if™ state-
ments that compare the prediction of the network with the threshold and that post
an exit at close order based on the comparison. The parameter thresh is stepped
from 50 to 80 in increments of 2.

RESULTS OF THE NEURAL EXIT TEST
Baseline Results
Table 15-l contains data on the baseline behavior of the MSES. The threshold
..was set high enough to prevent any net-based exits from occurring. The numbers
in this table are the same as those reported in Chapter 14 (Table 14-I) for an opti-
346



ma1 fixed stop and profit target. The abbreviations in Table 15-l may be inter-
preted as follows: SAMP = whether the test was on the training or verification
sample (IN or OUT); NETL = the total net profit on long trades, in thousands of
dollars; NETS = the total net profit on short trades, in thousands of dollars; PFAC
= the profit factor; ROA% = the annualized return-on-account; ARRR = the
annualized risk-to-reward ratio; PROB = the associated probability or statistical
significance; TRDS = the number of trades taken across all commodities in the
portfolio; WIN% = the percentage of winning trades; AVTR = the average prof-
it/loss per trade; and TRDB = the average number of bars or days a trade
was held.
There was great consistency between in- and out-of-sample performance:
The average trade lost $1,581 in-sample and $1,580 out-of-sample; both samples
had 39% winning trades; and the risk-to-reward ratios were - 1.46 in-sample and
- 1.45 out-of-sample.


Neural Exlt Porlfolio Results
Table 15-2 is the standard optimization table. It shows the in-sample portfolio per-
formance for every threshold examined and the out-of-sample results for the
threshold that was the best performer during the in-sample period.
In-sample, an improvement in overall results was obtained from the use of
the additional neural network exit. The average trade responded to the threshold in
a consistent manner. A threshold of 54 produced the best results, with an average
trade losing $832. There were 41% wins and an annualized risk-to-reward ratio of
-0.87. The numbers represent a dramatic improvement over those for the baseline
presented in Table 15-l. Out-of-sample, however, no improvement was evident:
Performance was not too different from that of the optimal MSES without the
neural signal element. In the tests conducted using the neural net for entries, per-
formance deteriorated very significantly when moving from in-sample to out-of-
sample data. The same thing appears to have happened in the current test, where
the same net was used as an element in an exit strategy.




Baseline Performance Data for the Modified Standard Exit Strategy
to Be Used When Evaluating the Addition of a Neural Forecaster
Signal Exit

SAW 1 NETI. 1 NETS 1 PFAC 1 RCA% / ARRR I PRO6 I l-R06 1 WIN% [ AVTR ITRDB
IN I -19761 -4073) 0.831 -10.31 -1.461 1.0000˜ 38261 391 -15811 6
OUT I -9741 -163zl 0.641 -21.61 -1.451 O.BS651 16491 391 -15501 8
341
CHAPTER 15 Adding AI to Exits



TABLE 15-2

Portfolio Performance of the Modified Standard Exit Strategy with an
Added Neural Signal Exit Evaluated over a Range of Threshold
Parameter Values




Neural Exit Market-by-Market RBBUitB

Table 15-3 shows the performance data for the portfolio, broken down by market,
for the optimal MSES with the added neural signal exit. The results are for tire
composite exit with the best threshold value (54) found in the optimization pre-
sented in Table 15-2.
Live Hogs was the only market that was substantially profitable in both sam-
ples. A number of markets (e.g., the Deutschemark and Japanese Yen) showed
strong profitability in-sample that was not evident out-of-sample. On the long side,
the NYFE and Unleaded Gasoline were profitable in both samples. This could eas-
ily be a statistical artifact since, in-sample, a large number of markets had prof-
itable performance on the long side.

TEST METHODOLOGY FOR THE GENETIC EXIT
COMPONENT
Since it is almost certain that different rules are required for the long side, as opposed
to the short side, two tests are rtm. In the first test, random entries are generated for
long positions using the standard random entry strategy. Any short trades generated
are simply not taken. Rules are genetically evolved for inclusion in the MSES for the
long positions. In the second test, only short entries are taken. Any long entries gen-
erated by the random entry strategy are ignored. An attempt is made to evolve rules
that work well as additional elements to the MSES for the short side.
342



TABLE IS-3

Market-by-Market Performance for the Modified Standard Exit with a
Neural Signal Addition Using Random Entries
345




The code above shows the logic of both the entries and exits. The modeltype
parameter controls whether the longs or shorts are tested. Parameters ptlim and
mmsfp are for the profit target and stop (respectively). They are fixed at the same
optimal values used in the neural network test earlier, Each of the three rules is cal-
culated as a series of TRUE/FALSE values, and if all three rules are TRUE, a rule-
based exit signal (exitrig) is generated. In the exit code, “if™ clauses have been
added. For example, an exit at the close is generated ifan exit signal is produced
by all three rules being TRUE (exitsig = TRUE). The evolution of rules for the
long and short sides follow the same steps as in the chapter on genetics, in which
similar rules were evolved for use in entries. The same 12 parameter chromosomes,
broken into three genes (each specifying a rule), are employed, and there is no
change in that logic in the current test, Rules for the short side and for the long
side are produced by allowing 2,500 generations to pass (2,500 runs using
OptEvolve). The top 10 solutions for the longs and for the shorts are then tested
on both the in-sample and out-of-sample data.
346



Top 10 Solutions with Baseline Exit
Table 15-4 shows the top 10 solutions found for the long side and for the short
side. In the table, LINE = the line or generation number; PROB = the probabili-
ty or statistical significance (the decimal point is omitted but implied in the for-
matting of these numbers); $TRD = the average dollars-per-trade; TRDS = the
total number of trades taken; PFAC = the profit factor; and AROA = the annual-
ized return-on-account.
The best solution for the longs was discovered in the 845 generation of the evo-
lutionary process. For the short side, it was in the 1,253 generation. ln contrast to the
situation when rules were evolved for use in an entry model, none of the solutions
were profitable. However, Table 15-5 shows that when genetically evolved rule-based
exits were added, substantial improvement over the baseline was achieved.
The rules in Table 15.4 were translated into plain language. The rules for
exiting a long position were as follows: If the close on the current bar is greater




TOP I 0 Solutions from the Evolutionary Process
for Longs and for Shorts
than a 12.bar exponential moving average (EMA) of the closes, but is less than a
49-bar EMA of the closes, and the current bar represents a 6.bar new high, then
exit the long trade. The rules seem to be searching for a situation in which the
longer trend is down, but a short-term retracement against the trend has occurred
and has reached a point where completion of the retrxement is likely and the
longer-term downward trend will resume-a sensible point to exit a long position.
The rules for the short side suggest that an exit should occur if the close on the
current bar is greater than the 16-bar EMA of the closes and a 22.bar simple mov-
ing average of the closes, and if the MACD is sloping down. The specific MACD
used employs a 6-bar EMA for its faster moving average and a lo-bar EMA for
its slower moving average. The idea encapsulated in these rules seems to be that
it is wise to close out short positions if the market, when smoothed, still appears
to be moving down, but the most recent close broke above two moving averages,
indicating the market may be starting a new trend up.


Results of Rule-Based Exits for Longs and Shorts
Table 15-5 presents the performance data for best of the top 10 solutions (longs
and shorts) for the MSES with the addition of genetically evolved, rule-based
signal exits. Trades were entered randomly. The table is broken down into results
for long positions and results for short positions. It is further broken down by sam-
ple and test. Sample (IN or OUT) and test may be BSLN (when the rules were not
used) or RULE (when the mles were. used).
On the long side, in-sample, the addition of the genetically evolved rule-based
exit substantially reduced the loss on the average trade from a baseline of $688 to $324.
The percentage of winning trades increased from 41 to 43%. The annualized risk-to-


TABLE 19-B

Performance of the Modified Standard Exit Strategy with an Added
Genetically Evolved Rule-Based Signal Exit When Trades Are
Entered Randomly
reward ratio improved from -0.35 to -0.17. Out-of-sample, the benefit of the genet-
ically evolved mle-based signal exit was maintained, but not quite as dramatically. The
loss on the average trade was cut from $1,135 to $990. The percentage of winning
trades increased from 39 to 41%. The risk-to-reward ratio improved slightly from
-0.61 to -0.60. Overall, adding the genetically evolved rule-based element to the exit
strategy worked. In contrast to the neural exit, the benefit was maintained out-of-mm
ple, suggesting that curve-fitting and over-optimization were not major issues.
On the short side, similar benefits were observed in both samples. In-sample,
the addition of the genetically evolved rule-based element reduced the loss on the
average trade from a baseline of $2,084 to $1,645. The percentage of winning
trades remained unchanged. Paradoxically, the annualized risk-to-reward ratio
worsened somewhat, going from - 1.09 to - 1.15. Out-of-sample, the loss on the
average trade dropped substantially, from $1,890 in the baseline test to $1,058,
when the rule element was active. The percentage of winning trades rose from 38
to 40%, and the annualized risk-to-reward ratio improved, from - 1.02 to -0.73.
Again, the addition of the genetically evolved rule-based signal exit to the MSES
worked and continued to work out-of-sample.

Market-by-Market Results of Rule-Based Exits
for Longs
Table 15-6 contains information regarding the market-by-market performance of
the MSES for the long side, with the added rule-based signal exit. Several markets
showed profitability in both samples: the NYFE, Light Crude, Unleaded Gasoline,
and Live Hogs. Other markets were profitable in-sample, but lost heavily out-of-
sample (and vice versa). The consistency between in-sample and out-of-sample
results was not high.

Market-by-Market Results of Rule-Based Exits
for Shorts
Table 15-7 shows the same market-by-market breakdown as in Table 15-6, but
only the short side is represented. More consistency was evident between in-sam-
ple and out-of-sample performances for the short side than for the long. Most
notably profitable in both samples was the Japanese Yen. Light Crude, Unleaded
Gasoline, Feeder Cattle, Live Hogs, Soybean Meal, and Coffee were also prof-
itable in both samples.

CONCLUSION
Several important points were demonstrated by the above tests. First, neural net-
works hold up less well in out-of-sample tests than do genetically evolved rule-
based solutions. This is no doubt a result of the greater number of parameters
Market-by-Market Performance of the Modified Standard Exit with an
Added Genetically Evolved Rule-based Signal Exit When Tested
Using Random Long Trade Entries




involved in the neural network model, as compared with the rule-based models
being used. In other words, the effects of curve-fitting were damaging to the neur-
al network solution. Also discovered was the fact that the addition of a sophisti-
cated signal exit, whether based on a neural net or a set of genetically evolved
entry rules, can greatly improve an exit strategy. When the more robust, geneti-
cally evolved rules were applied, the performance benefits obtained persisted in
out-of-sample evaluations.
The neural network and the rule templates (but not the actual rules) that were
used in developing the signal exits were originally developed for inclusion in an
350




TABLE 15-T

Market-by-Market Performance of the Modified Standard Exit with an
Added Genetically Evolved Rule-Based Signal Exit When Tested
Using Random Short Trade Entries




entry model. When the rules were used in an entry model, it was acceptable for
them to generate rare event trades. In an exit strategy, however, rules that tire more
frequently would be more desirable. There is every indication that a set of rule
templates (and ways of combining the rules to obtain signals), specifically
designed for use in an exit strategy, would provide much better results than those
obtained here. The same should be true for neural networks.
WHAT HAVE WE LEARNED?

Curve-fitting can be bad not only when building entries, but also when
n

building exits.
n Sophisticated technologies, including genetic algorithms, can be effec-

tively used to improve an exit strategy.
. Even crude efforts to improve exits, such as those carried out here, can
enhance profits by several hundred dollars per trade.
Conclusion




A long road has been traveled since beginning the study of entry and exit strate-
gies. Sometimes the trip has been tedious and discouraging; at other times, stimu-
lating and surprising. As usual after extended journeys, the questions “What
knowledge has been gained?” and “How may that knowledge be applied?™ beg to
be answered. The first question will be addressed by a successively more detailed
examination of the results: going from discoveries made about the portfolio per-
formance of entire classes of models, to more specific model-order combinations,
down to an inspection of individual markets and how they are best traded.
The perspectives taken in the following discussions of what has been
achieved are analogous to views from an airplane flying at night. At first, the plane
is at a very high altitude: All that can be seen when looking down are large patch-
es of darkness (classes of models that are ineffective or lose) and some patches of
light (classes of models that, overall, work fairly well or, at least, perform better
than chance). This view provides a basic idea of which models are, overall, viable
relative to the entire portfolio of tradables.
The plane then descends. More detail is seen. It becomes evident that the
brightest spots are often formed by clusters of light having various luminosities
(model-order combinations that are, to one extent or another, profitable).
Occasionally the dark patches also contain small isolated points of brightness
(successful model-order combinations amid approaches that usually are ineffec-
tive). At this level, a number of dim areas can be seen as well (model-order com-
binations that are not profitable, but that have better than chance performance that
could be enhanced if combined with a good exit).
Finally, landing is imminent. It is possible to look inside the bright spots and
see their detail, i.e., the individual markets the various model-order combinations
trade best. The second question above can now be addressed: By identifying the
consistently profitable (across samples) model-order combinations and the mar-
kets best traded by them, a good portfolio trading strategy can be developed. At
this time, it will become clear that out of all the studies performed during the long
trip, enough has been learned to assemble a lucrative portfolio of systems and trad-
ables. By way of demonstration, such a portfolio will be compiled and run with
the standard exit strategy.

THE BIG PICTURE
For this perspective, each class of entry model (e.g., all trend-following moving
average models, all breakouts, all small neural networks) was examined in its
entirety. All the tests for each model type were averaged. Out-of-sample and
in-sample performances were separately evaluated.
By far the best out-of-sample performer was the genetic model: Out of all the
models, it was the only one that showed a substantial profit when averaged over
all the different tests. The profit per trade was $3,271.
Next best, in terms of out-of-sample behavior over all tests, were the small
neural networks. The neural network models were broken down into those for
small and large nets because curve-fitting appeared to be a serious issue, especial-
ly affecting large nets. The breakdown was a natural and easy one to accomplish
because, in the tests, each model was tested with one small and one large network.
Out-of-sample, all the small neural networks taken together averaged a loss of
$860 per trade. This indicates a significantly better than chance entry in that ran-
dom entries produced an average loss of over $2,000, with a standard deviation of
slightly under $400.
Going down in quality, the next best overall approach involved seasonal&y.
Altogether, all tests of seasonality models showed an average loss of $966 per trade.
Three of the moving average models (crossover, slope, and supportIre&
tance) followed the performance of seasonality. These models, when averaged
across tests, lost around $1,500 per trade, which is close to the $2,100.per-trade
loss expected when using random entries. In other words, the moving average
models were only marginally better than random.
All the remaining models tested provided entries that were very close to ran-
dom. Cycles were actually worse than random.
In-sample, the genetic models ($12,533 per trade), all the neural network
models (small, $8,940, and large, $13,082), and the breakouts ($1,537) traded
profitably. Out-of-sample, the genetic models continued to be profitable, the nets
were better than chance (although there was significant shrinkage in their perfor-
mance due to curve-fitting), and the breakouts deteriorated to chance (optimiza-
tion cannot be a factor in this case).
The next best performers in-sample were the support/resistance moving average
models ($300 loss per trade) and the seasonality models ($671 loss per trade).
Further down the ladder of best-to-worst performers were the lunar and solar
models, which lost $1,076 and $1,067, respectively. Losses in the $1,300 to
$1,700 range were observed for the moving average models. The oscillator and
cycle models exhibited losses of over $2,000 per trade; when taken as a whole,
these models were no better than chance.
It is interesting that the genetic model and the small neural network models
were the ones that held up out of sample. Such models offer great opportunities
for curve-fitting and tend to fail in out-of-sample tests and real trading.
Seasonality, which is only rarely the topic of articles, also exhibited potential. On
the other hand, the most popular methods (e.g., moving average, oscillator, and
cycle models) performed the worst, trading badly both in- and out-of-sample. The
breakout models are noteworthy in the sense that, taken as a group, they worked
well in the past; but, due to increased market efficiency, have since deteriorated to
the point where they currently perform no better than chance.
Table C-l contains the annualized return-on-account (the first line of each model-
order combination) and average dollars-per-trade (the second line of each model-order
combination) data for all the tests (all model and order combinations) conducted for entry
models using the standard exit strategy. The data presented arc for portfolio performance
as a whole. The model descriptions (leftmost column) are the same as used elsewhere in
this book. The last six lines of the table contain data that can be used as a baseline against
which the various entry models can be compared. The baseline data are derived from
using the random entry strategy with the unmoditied standard exit strategy. Mean ROA%
is the average return on account over several sets of random entries; StdLh ROA% is the
standard deviation of the return-on-account. Mean $TRD is the average dollars-per-trade
over several sets of random entries; and StdDev $TRD is the standard deviation of the
dollars-per-trade.
Breakout models had the unique characteristic of being consistently profitable
in-sample across almost every model-order combination tested. Except for the
volatility breakouts, these models performed much better than chance, albeit not
profitably, out-of-sample; i.e., the loss per trade was under $1,000, sometimes under
$300 (the average loss to be expected was around $2,000 with random entries). In
other words, the breakouts, taken together, were better than random. However, out-
of-sample, the volatility breakouts did much worse than chance: With at-open and
on-stop orders, the model lost more than $5,000 per trade, as though the market™s
current behavior is precisely designed to make these systems costly to trade.
The trend-following moving average models (crossover and slope) all per-
formed slightly better than chance in-sample: They all lost rather heavily, but the
losses were almost always less than $2,000. None of the systems, however, did
very well. Out-of-sample, the picture was somewhat more variable, but had the
same flavor: Most of the models were somewhat better than chance, and one or
two were much better than chance (but still not profitable).
356



TABLE C-l

Summary of Portfolio for All Entry Models Tested with All Order Types
Crossoverwith -10 -7 6 -14 -21 20 -4 -5
-1512 -3408 1877 -364 - 1 0 8 1
conflmation -1185 -032 846

-20 -23 1 -7 -14
Crossover with -10 -3 -2
TABLE C-l

(Continued)
The countertrend moving average models were more variable than the trend-
following ones. Many of them showed much smaller losses or even small profits,
in-sample. A similar picture was seen out-of-sample, especially with the simple
moving average support/resistance model.
Except for the MACD divergence model, which behaved differently from the
others, oscillators performed very poorly. There was a lot of variability, but on the
whole, these models gave per-trade profits that were worse than expected by
chance both in-sample and out-of-sample. The RSI overbought/oversold model
was the worst of them all. In both samples, it provided staggering losses that were
(statistically) significantly worse than those that would have been achieved with a
random entry.
The seasonal models, on the whole, were clearly better than chance. Although
only one of these models actually provided profits in both samples, two of them
were profitable out-of-sample, and several had only very small losses (much less
than would be expected by chance using random entries) across samples.
The basic lunar model had mixed findings. Most of the in-sample results
were slightly positive when compared with chance (the random entry), but not
profitable. The basic crossover model, however, was decidedly biased above
chance in both samples.
Although the solar models performed slightly better than chance in-sample,
they were mixed and variable out-of-sample. This was also true for the cycle models.
However, the cycle models, when using entry at open or on limit, actually performed
significantly worse in recent years than a random entry. As with breakouts, the find-
ings are not due to optimization; significant curve-fitting was only detected with the
genetic and neural network models. Because of the tremendous sample involved in
the portfolio, the optimization of one or two parameters, necessary for most models
(other than the genetic and neural ones), had minimal curve-fitting effect.
Surprisingly, the neural network models showed a fairly consistent bias to per-
form better than chance out-of-sample. In-sample, of course, performance was stel-
lar across all tests. There was shrinkage (evidence of curve-fitting), but the shrinkage
was not complete, leaving some predictive utility in the out-of-sample data.
The results for the genetically evolved rules were the best. In-sample, per-
formance was excellent, Out-of-sample, performance was exceptional for models
involving long positions.

Summary
Many of the models described as significantly better than chance (i.e., better than
what would be produced by a random entry) would likely become profitable if cou-
pled with a better exit strategy. In Part III, it was evident that when tested with ran
dom entries, the use of a good exit could bolster profits (or cut losses) by about $1,000
per trade. This means that, with a good exit, some of the entry models that had loss-
es of several hundred dollars could be brought into positive, profitable territory.
CO”Cl”Si””




As mentioned above, the journey was a long one, sometimes tedious and dis-
couraging. However, this birds-eye view revealed that a lot of potentially profitable
entry models were indeed discovered. There were also a number of surprises: Despite
terrible reputations and dangerous tendencies toward curve-fitting, the neural network
and genetic models were the best performers when tested with data that was not used
in training or evolving. Another surprise was that some of the most popular trading
approaches, e.g., moving-average crossovers and oscillator-based strategies, turned
out to be among the worst, with few exceptions. The results of the cycle models were
also revealing: Because of their theoretical elegance, better-if not ideal-perfor-
mance was expected. However, perhaps due to their popularity, poor performance
was observed even though the implementation was a solid, mathematical one.


POINTS OF LIGHT
The portfolio performance of each model was examined for each of the three order
types (at open, on limit, and on stop). Out-of-sample and in-sample performances
were separately evaluated.
The out-of-sample performance was, by far, the best for the long-side genet-
ic models. Entry at open was especially noteworthy, with a 64.2% in-sample return
and 41.0% out-of-sample. The same model was also profitable with entry on limit
and on stop, yielding very high dollars-per-trade profits, although small numbers
of trades (easily increased with more elaborate models of this kind).
In terms of out-of-sample performance, the next best specific model-order
combination was the seasonal crossover with confirmation using entry on stop. Like
the long genetic models, this one was significantly profitable in both sampling peri-
ods: in-sample, $846 per trade, with a 7.4% return; out-of-sample, $1,677 per trade,
with a 9.5% return. Other seasonal models also did okay, with out-of-sample profits
being made using the simple seasonal crossover model,
Next down the list was the short turning-point model that used the small 16-
10-1 network. This model was profitable in both samples across all orders: out-of-
sample, at open, a 9.3% annualized return, with $580 per trade; in-sample, a
35.2% return and $8,448 per trade profit.
While still on the subject of neural networks, the reverse Slow %K model
was a profitable performer, especially when a stop order was used. Out-of-sample,
the annualized return was 6.1%, with $362 per trade. In-sample, there was a 22.5%
return, with a $6,764per-trade profit. Note the large shrinkage from in-sample to
out-of-sample for these models: While this is evidence of curve-fitting, enough
real curves were caught for profits to be made in the verification sample.
Another model that produced profits in both samples was the MACD diver-
gence model, especially with entry on limit. This model had a 6.1% annualized
return, out-of-sample, and a $9X5-per-trade profit. In-sample, the figures were a
6.7% return and a $1,250 profit per trade.
361
Conclusion




Finally, among the models that were profitable in both samples was the sim-
ple moving-average suppotVresistance model with entry on stop: It took 6.4% out
of the market, with $482 per trade, out-of-sample; and $5.8%, with $227 per trade,
in-sample.
Almost all other models lost money out-of-sample, and often in-sample, as
well, The only exception was the volatility breakout model restricted to the curren-
cies, which performed fairly well. Out-of-sample, it had an 8.5% return and made
$2,106 per trade. In sample, it had a 12.4% return, with $3,977 profit per trade.

Summary
Even though most of the other model-order combinations lost out-of-sample, in
many cases, the losses were much less than would be expected with a totally ran-
dom entry. In a number of instances, however, they were worse.
It seems evident that there are a number of models that, although not ideal
and in need of further development, do yield profitable trading that holds up in a
verification sample and yields reasonable statistics.

LOOKING INTO THE LIGHT
Until this point, only tests and models that operate on the whole portfolio have
been discussed. In the course of the tests, many observations were made regarding
the performance of specific models when trading individual markets. A recurrent
observation was that certain models seem to trade certain markets well, while
other models trade them poorly. Some markets just seem to be difficult to trade,
regardless of model. There is no doubt that by selecting a number of the better
models, and then selecting markets that these models trade well, a portfolio of sys-
tems to trade a portfolio of markets could be assembled. Therefore, good system-
market combinations were selected from among the tests conducted in this book.
No optimization of model parameters was performed.
A portfolio was assembled on the basis of in-sample statistical significance.
The intention was to find one good model-order combination for each of the mar-
kets in the portfolio. If there were several potential models for a given market, the
additional ones were discarded based on such things as model complexity (the
more complex the model, the less it was trusted), mediocre portfolio performance,
and other similar factors. The specific model-order combinations spanned the
entire spectrum of models and orders tested, with various oscillators, moving aver-
ages, lunar and solar models, and seasonal and neural network models being rep-
resented; genetic models, however, were not included. In the current tests, the
particular genetic models that were evolved only traded rare events. For those in-
sample markets that performed well, there were generally no out-of-sample trades.
The profitable out-of-sample behavior was achieved on almost a totally different
362



FIGURE C-l

Equity Growth for Multiple-System and Market Portfolio




set of markets than the in-sample behavior of the model. This does not mean that
the out-of-sample performance was bad while the in-sample performance was
good, but rather that most markets simply did not trade in one sample if they did
in the other. The low number of trades observed with the genetic models was due
to the specific namre of the particular rule templates and the ways in which the
individual rules were combined to obtain buy and sell signals. With some changes
in the rule templates, especially in the number of rules used and in how they are
combined, the pattern of rare event trading can be entirely altered.
There were times when the preferred kind of model was not available for a
given market. In such cases, models were examined that performed poorly on a
whole-portfolio basis, but that did trade one or two difficult markets acceptably.
For example, the RSI overbought/oversold model with entry on limit was a poor
performer on a portfolio-wide basis. However, this model traded Gold and Silver
reasonably well. It pulled returns of 27.3 and 3.9%, annualized, on the in-sample
data, with average trades of $9,446 and $4,164, respectively. Out-of-sample, the
system pulled 23.6% out of the Gold and 51.7% out of the Silver markets, with
average trades yielding $12,194 and $24,890, respectively.
One of the large neural networks that appeared to be highly over-optimized
was used for the three wheat markets-markets that did not trade at a statistically
significant level with any of the other models. The large, long-side, turning-point
network with entry on limit, however, had high statistical significance when trad-
ing each of the wheats, pulling more than 40% annually from each, and more than
$15,000 per trade. The amazing thing is that, out-of-sample, despite the size of the
net and the degree of curve-fitting seen on its portfolio performance, the model
pulled in no less than 24%, with $5,000 per trade, from each of the wheats.
The cycle model, which worked well on hardly any market, did trade the
S&P 500 profitably-returning 15.3%, with an average in-sample trade of $4,613,
and 21.4% with $4,698-per-trade profit out-of-sample. It should be noted that a
cycle model was found to trade the S&P-500 successfully in the tests reported in
our earlier study (Katz and McCormick, May 1997).
Once each market was paired with a good model-order combination, the per-
formance data were analyzed, both in- and out-of-sample, for each of the markets.
An equity curve was prepared that covered both periods (see Figure C-l). Returns
and statistical significance were calculated for the multiple-model portfolio, both
in-sample and out-of-sample. It was surprising to discover that the out-of-sample
performance data revealed a return-on-account of 625% annualized! A manifesta-
tion of the Holy Grail? Because model-market combinations were selected on the
basis of their in-sample statistical significance, the 544% annualized in-sample
return was not unexpected. The probability of obtaining an in-sample profit as
large as that is less than 1 in 3,000,000,000,000,000,000 (i.e., 3 X lo™*). Even if
massive amounts of optimization, with tests of tens of thousands of combinations,
took place, the results would still be extremely significant, in a statistical sense.
Out-of-sample, the probability of finding a risk-to-reward ratio or annualized
return as good as that observed is less than 1 in 40 million. Again, even corrected
for extensive optimization, the results would still be of extreme statistical signifi-
cance. In fact, no out-of-sample optimization took place. In-sample, the systems
were only optimized on the entire portfolio. Model parameters were never adjust-
ed for the selected markets on which the models were to be traded. And only the
minimal standard exit strategy was used. Performance could be very substantially
improved using the best of the exits found in Part III.
These findings demonstrate that while most systems do not work and most
tests show losses, a sufficiently extensive search (as conducted in this book) can
discover enough that do work to put together a portfolio trading strategy capable
of producing nothing less than stellar results.
Conclusion
364




COMPANION SOFTWARE AVAILABLE
We invite all readers to visit our website at: www.wientiLic-consultants.com or
to e-mail us at kat.z@scientl6c-consultants.com.
Those who wish to replicate and expand on our research may obtain afree
copy of the C-Trader Toolkit (the software. required to run the code presented in
this book) from our website at www.s&nGfic-eonsultants.com. A CD-ROM is
also available for the nominal cost of $59.00. It contains the following:
m Complete code for every method tested in this book
. Commodities data from Pinnacle
. Spreadsheets containing all optimization data, market-by-market analy-
ses, equity cunw, figures, and tables
m The C-Trader Toolkit, which includes the C+ + Trading Simulator,
OptEvolve (the genetic optimizer), the Portfolio Simulation Shell, and
related manuals
NtlllE Company
Address
City state ˜ ZP
C”U”try Country code
Phone: home (-) office (-)
E-mail
Fax (-1

The companion CD-ROM: $59.00 X ˜ copies $
Numerical Recipes in C:
994 page book $54.95 X ___ copies
software IBM disks $39.95 X ˜ copies

SHLPPlNG g. HANDLING:
for CD only: add $3.50 per copy US, $7.50 outside US $
for Numericul Recipes: add $12 US, $35 outside US $
SALES TAX
(NYS residents add -% for your county 1s
TOTAL $
CHECK ONE:
_ Enclosed is my check “I money order (U.S. only)
_ charge my ldkuMas˜ard/AmEx acwunt (ml in inf”nnati”” below)
acc”““t # expiration
signature
E-mail your order (katz@sclent&-consultants.com), “I mail, phone, “ I fax it to:
SCIENTIFIC CONSULTANT SERVICES, INC. Phone & fax:
20 Stagecoach Road, Selden, New York 11784 631-696-3333
APPENDIX


References and Suggested
Reading




Alexander, Colin (June 1993). “Trade with Moving Averages.” Technical Anolysir of Stockr a n d
Commodities, pp. 61-7 I.
Appel, Gerald (1990). The Advanced Moving Average Convergence-Divergence Trading Method.
Videotape and manual distributed by Signalert Cotportion, New York (516-829-6444).
Bartie, Scott (September 1996). “The COT Index.” Technical Analysis of Stocks and Commodities,
pp. 16-36.
Barr&, Scott (October 1996). “Pork Bellies and the CtX Index.” Technical Analysis ofStocks und
Commodities, pp. 79-92.
Bernstein, Jake (1995). Trade Your Way fo Riches. MBH Commodity Advisors, Inc. (l-800-457-
0X25), 1995.
Blau, William (January 1993). “Stochastic Momentum:™ Technical Analysis of Stocks and
Commodities. pp. 26-35.
Burke, Gibbons (May 1993). “Good Trading a Matter of Breeding?” Futures Magazine, pp. 26329.
Center for Solar and Space Research, Yale University (1997). Sunspot Predictions. Release distib-
uted by Virtual Publishing Company.
Chande. Tushar S. (March 1992). ˜Adapting Moving Averages to Market Volatility.” Technical
Analysis of Stockr and Commodities, pp. 4653.
Davies, D. W. (June 1993). “Cyclical Channel Analysis and the Commodity Channel Index.”
Technical Analysis of Stocks and Commodities, pp. 3845.
Davis, Lawrence (Ed.) (1991). Handbook of Genetic Algorithms. New York: Van Nostrand Reinhold.
Ehlers, John (March 1989). “Moving Averages and Smoothing Filters,” Technical Analysis of Stocks
and Commodities, pp. 4246.
Gauquelin, H., Gauquelin, R., and Eysenck, S. B. G. (1979). “Personality and Position of the Planets
at Birth: An Empirical Study.” British Journal of Social and Clinical Psychology, Vol. 18, pp.
71-75.
Goedde, Richard (March 1997). “Timing a Stock Using the Regression Oscillator.” Technical
Analysis of Stocks and Commodities, pp. 54-60.
Hannula. Hans (November 1991). “The Seasonal Cycle.” Technical Analysis of Stocks and
Commodities, pp. 6548.
Hoel, Paul G. (1966). Elemental Statiufics, 2d ed.. New York: John Wiley & Sons.
365
366



Holland, John (1975). A d a p t a t i o n i n Narural a n d Arfificial Systems. Ann Arbor: The University of
Michigan Press.
Jurik, Mark (1999). “Finding the Best Data.” Computerized Trading. Mark Jurik (Ed.). New York:
New York Institute of Finance/Prentice Hall, pp. 355-382.
Katz, Jeffrey Owen (April 1992). “Developing Neural Network Forecasters for Trading.” Technicul
Analysis of Stocks and Commodities, pp. 5848.
Katz, Jeffrey Owen, and McCormick, Donna L. (1990). Calendar Effects Chart. New York:
Scientific Consultant Services.
Katz. Jeffrey Owen, and McCormick, Donna L. (March/April 1993). “Vendor™s Forum: The
Evolution of N-TRAIN.” PCAI, pp. 4&46.
Katz, Jeffrey Owen, and McCormick, Donna L. (1994). “Neural Networks: Some Advice to
Beginners.” Trader™s Cat&g and Resource Guide, Vol. II, No. 4, p. 36.
Katz, Jeffrey Owen. and McCormick, Donna L. (July/August 1994). “Neurogenetics and Its Use in
Trading System Development.” NeumVe$t Joumnl, pp. X-1 1 .
Katz, Jeffrey Owen, and McCormick, Donna L. (1995a). “Introduction to Artificial Intelligence:
Basics of Expert Systems, Fuzzy Logic, Neural Networks, and Genetic Algorithms.” Virlual
Trading, I. Lederman and R. A. Klein (Eds.). Chicago: Probus P u b l i s h i n g , pp, 3 - 3 4 .
Katz, Jeffrey Owen, and McCormick, Donna L. (1995b). “Neural Networks in Trading.” Vinual
Trading, J. Lederman and R. A. Klein (Ed%). Chicago: Pmbus Publishing, pp. 3544.
Katz, Jeffrey Owen, and McCormick, Donna L. (November 1996). “On Developing Trading
Systems.” Technical Analysis of Stocks and Commodities, pp. 46-60.
Katz, Jeffrey Owen, and McCormick, Donna L. (December 1996). “A Rule-Based Approach to
Trading.” Technical An˜alysis of Stocks an,d Commodities, pp. 22-34.
Katz, Jeffrey Owen, and McCormick, Donna L. (January 1997). “Developing Systems with a Rule-
Based Approach.” Technical Analysis of Stocks and Commodiries, pp. 38-52.
Katz. Jeffrey Owen, and McCormick, Donna L. (February 1997). “Genetic Algorithms and Rule-
Based Systems.” Technical Annlycis of Stocks and Commodities, pp. 46-60.
Katz, Jeffrey Owen, and McCormick, Donna L. (April 1997). “Seasonality and Trading.” Technical
Analysis of Stocks and Commodities, pp. 50-61.
Katz, Jeffrey Owen, and McCormick, Donna L. (May 1997). “Cycles and Trading Systems.”
Technical Analysis of Stocks and Commoditie$ pp. 38-46.
Katz, Jeffrey Owen, and McCormick, Donna L. (June 1997). “Lunar Cycles and Trading.” Technical
Analysis of Stocks and Commodities, pp. 3846.
Katz, Jeffrey Owen, and McCormick, Donna L. (July 1997). “Evaluating Trading Systems with
Statistics.” Technical Analysis of Stocks and Cornmodifies, pp. 50-61.
Katz, Jeffrey Owen, and McCormick, Donna L. (August 1997). “Using Statistics with Trading
Systems.” Techniccll Analysis rfSfockr und Cornmodifies, pp. 32-38.
Katz. Jeffrey Owen, and McCormick, Donna L. (September 1997). “Sunspots and Market Activity.”
Technical Analysis of Stocks and Commodities pp. 46-54.
Katz, Jeffrey Owen, and McCormick, Donna L. (November 1997). “Adding the Human Element to
Neural Nets.” Technical Analysis of Stocks and Commodities, pp. 5264.
Katz, Jeffrey Owen, and McCormick, Donna L. (Febmay 1998). “Exits, Stops and Strategy.”
Technical Annlyris of Stocks and Commodities pp. 32-40.
Katz, Jeffrey Owen, and McComdck, Donna L. (March 1998). “Testing Exit Strategies,” Technical
Analysis of Srocks and Commodities, pp. 35-42.
Katz, Jeffrey Owen, and McCormick, Donna L. (April 1998). “Using Trailing Stops in Exit
Strategies.” Technical Analysis @Stocks and Commodities, pp. 86-92.
Katz. Jeffrey Owen, and McCormick, Donna L. (May 199X). “Using Barrier Stops in Exit
Strategies.” Technical Ana1,ysi.r of Stocks and Commodin™es, pp. 63-89.
Katz, Jeffrey Owen. and McCormick, Donna L. (July 1998). “Barrier Stops and Trendlines.”
Technical Analysis of Stocks and Commodities, pp. 44-49.
Katz, Jeffrey Owen, and McCormick, Donna L. (1999). “Case Study: Building an Advanced Trading
System.” Computerized Truding, Mark Jurik (Ed.). New York: New York Institute of
Finance/Prentice Hall, Pp. 317-344.
Katz, Jeffrey Owen, and McCormick, Donna L. (February 1999). “Trading Stocks with a Cyclical
System.” Technical Analysis of Stocks and Cornmodifies, Pp. 3&42.
Katz, Jeffrey Owen, and Rohlf, F. James (April 1975). “Primary Roduct Functionplane: An Oblique
Rotation to Simple Structure.” Journal of Multivariare Behavioral Research, Vol. 10, pp.
219-232.
Knight, Sheldon (September 1999). “How Clean Is Your End-of-Day Data?” Futures Magazine, p.
64.
Kmtsinger, Joe (I 994). The Trading Systems Toolkit, Chicago: Probus Publishing.
Lederman. J., and Klein, R. A. (Eds.). (1995). Virrunl Trading. Chicago: Probus Publishing.
Technical Analysis of Stocks
Lupa. Louis M. (December 1994). “Trading Markets with Stochastics.”
and Commodities, pp. 38-49.
Marder, Kevin (1999). “Financial Data Sources.” Computerized Trading, Mark Jurik (Ed.). New
York: New York Institute of Finance/Prentice Hall, pp. 345-354.
Masters, Timothy (1995). Neural, Novel & HybridAlgorithmsfor lime Series Prediction. New York:
John Wiley & Sons.
Mayo, J., White, 0.. and Eysenck, H. J. (1978). ˜An Empirical Study of the Relation between
Astrological Factors and Personality.” The Journal of Social Psychology, Vol. 105, pp.
229-236.
McWhorter, W. Lawson (January 1994). “Price/Oscillator Divergences.” Technical Analysis of
Stocks and Commodities, pp. 95-98.
Meibahr, Stuart (December 1992). “Multiple Length Stochastics.” Technical Analysis of Sroch and
Commodities, pp. X-32.
Meyers, Dennis (May 1997). “Walk Forward with the Bond XAU Fund System.” TrchnicalAnalysis
of Stocks and Commodities, pp. 16-25.
Montgomery, Douglas C., and Peck, Elizabeth A. (1982). Inrroduction fo Linear Regression
Analysis. New York: John Wiley & Sons.
Mulloy, Patrick G. (Febmary 1994). “Smoothing Data with Less Lag.” Technical Analy,ris of Stocks
and Commodities, pp. 58-70.
Murphy, John I. (1991). Intermarkef Technical Atudy.ris: Trading Strategies for the Global Stock,
Bond, Commodity and Currency Markets. New York: John Wiley & Sons.
Myers, Raymond H. (1986). Classical and Modem Regression with Applications. Boston: Duxbury
press.
Oliver, Jim (March 1994). “Finding Decision Rules with Genetic Algorithms.” AI Experr, pp, 32-39.
Pardo, Robert (1992). Design, Tesfing, and Optimizafion of Trading Sysfems. New York: John Wiley
& Sons.
Press, W.H., Flannery, B.P., Teukulsky, S.A., and Vetterling, W.T. (1986). Numerical Recipes: The
Arf of Scientific Computing. Cambridge, England: Cambridge University Press.
Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P. (1992). Numerical Recipes in C.
Cambridge, England: Cambridge University Press.
Price, Kenneth, and Storm, Rainer (April 1997). “Differential Evolution.” Dr Dobbs Journal. p p .
18-24.
Ruggiero, Murray A., Jr. (April 1994). “Getting the Lag Out.” Futures Magazine, pp. 4-8.
Ruggiero, Murray A., Jr. (October 1996). ˜Trend-Following Systems: The Next Generation.” Futures
Magazine.
368 APPl3NDlx Reference and Suggested Reading



Ruggiero, Murray A., Jr. (1997). Cybernetic Trading. New York: John Wiley & Sons.
Rug&m, Murray A., Jr. (May 1998). “Unholy Search for the Grail.” Funcres Magazine.
Schwager, Jack (October 1992). “Selecting the Best Futures for Computer Testing.” Technical
Analysis of Stocks and Commodities, pp. 65-71.
Sharp+ William E (Fall 1994). ˜The Shape Ratio.” Journal of Porffolio Management.
Space Science Institute (1996). ˜A Magnetic Storm Rips through !&ah™s Atmosphere.” A news
release on their web site: www@www-ssi.colorado.edu.
Star, Barbara (July 1993). “WI Variations.” Technical Analysis ofSlocks rind Commodities, pp. 5460.
Stendahl, David (1999). “Evaluating Trading Performance.” Computerized Trading. New Jersey:
New York Institute of Finance/Prentice Hall, pp. 137-162.
Sweeney, John (1993). “Where to Put Your Stops.” Technical Analysis of Srocks and Commodities
(Bonus Issue), pp. 3&32.
Sweeney, John (April 1998). “Applying Moving Averages.” Technical Analysis of Stocks and
Commodities, pp. 48-50.
Tilley, D.L. (September 1998). “Moving Averages with Resistance and Support.” Technical Analysis
of Srocks and Commodities, pp. 62-87.
Trippi, Robert R., and Turban, Efraim (Eds.) (1993). Neural Nenvorks in Finance and Investing.
Chicago: Pmbus Publishing.
White, Adam (April 1993). “Filtering Breakouts.” Technical Analysis of Sfocks and Commodities,
pp. 3w1.
Wilder, J. Welles (1978). New Concepfs in Technical Trading Systems. Trend Research.
Williams, Larry (1979). How Z Made One Million Dollars Last Year Trading: Commodities. New
York: Windsor Books.
Yuret, D&z, and de la Maza, Michael (June 1994). ˜A Genetic Algorithm System for Predicting the
OEX,” Technical Analysis of Stocks and Comnwdiries, pp. 5864.
INDEX




Breakout models (Conr,):
breakout type. 104

<<

. 8
( 9)



>>