Testing a moving average crossover system - Aussie Stock Forums

# Thread: Testing a moving average crossover system

1. ## Testing a moving average crossover system

Greetings all --

The thread that prompted me to make this posting began (on the AmiBroker Yahoo forum) with the question of whether a system based on a simple-moving-average crossover was profitable and would remain profitable.

I used this code to test that:
// Test MA Crossover.afl
//

ShortPrice = CoverPrice = C;

OptimizerSetEngine("cmae");

FastMALength = Optimize("FMAL",1,1,200,1);
SlowMALength = Optimize("SMAL",10,1,200,1);

FastMA = MA(C,FastMALength);
SlowMA = MA(C,SlowMALength);

Sell = Cross(SlowMA,FastMA);
Short = Sell;

e = Equity();

Plot(C,"C",colorBlack,styleCandle);
Plot(e,"equity",colorGreen,styleLine|styleOwnScale );

//////////////////////

Samantha suggests Buy when the monthly price is above the 10 month SMA, Go to cash when it is below.

When tested on the 500 stocks currently in the S&P 500, using end-of-day data from Quotes Plus that goes back to about 1993 (for companies with history that long), the median exposure is about 50% and the median RAR is about +9%.

As asked, the question assumes that being long or cash is the correct allocation. For many reasons, a trader may want to consider only long positions. But recent history (1984 to the present) has been a period of very strong rising markets. The future might be different. Using SPY as a surrogate for the broad market, the result of that same test are an exposure of 61% and an RAR of 12.6%.

1. Using full knowledge of the history, what are the best parameters for a trader who wants to use only long positions?

First, we need to decide how to measure best. We'll make a few runs with different objective functions and note the results.

Net % Profit.
The best values for the parameters are 2 and 12. Net profit is 304% (every initial \$1.00 becomes \$4.04). Maximum drawdown is 15.3%.

CAR/MDD.
Best values are 116 and 15. Backwards. This implies the trade should be flat when the faster moving average is above the slower, be long when it is below. Net profit is 51%. Exposure is 6%, RAR is 39%.

Maximum System Drawdown.
The best result is to never trade.

Sharpe Ratio -- used by many money management firms.
Best values are 97 and 13. Backwards again. 64% return with 13% exposure, for an RAR of 22.7%.

RRR (Same as K-Ratio).
Best values are 1 and 12. 292% return, exposed 63%, RAR of 13.8%.

The best results depend very much on the definition of best.

2. If the trader was able and willing to take short positions, what is best for them?

Net % profit.
Best values are 7 and 18. 64% return, exposed 26%, RAR of 11.7%.

CAR/MDD.
Best values are 1 and 24. 63% return, exposed 25%, RAR of 12.1%.

Maximum System Drawdown.
The best result is to never trade.

Sharpe Ratio.
Best values are 5 and 22. 63% return, exposed 24%, RAR of 12.5%.

RRR.
Best values are 7 and 18. 64% return, exposed 26%, RAR of 11.7%.

3. If the trader wants a system that is always exposed, long or short, and a single set of values that signal reversals.

Net % Profit.
Best values are 1 and 22. Net profit is 534%, exposure 88% (flat at the beginning until the averages have enough data), RAR of 13.6%. Maximum drawdown is 16.3%.

CAR/MDD.
Best values are 1 and 23. Net profit is 534%, exposure 88%, RAR 13.6%.

Maximum System Drawdown.
The best result is to never trade.

Sharpe Ratio -- used by many money management firms.
Best values are 78 asnd 47. Backwards again. 76% return with 34% exposure, for an RAR of 10.3%.

RRR (Same as K-Ratio).
Best values are 7 and 18. 522% return, exposed 89%, RAR of 13.2%.

However --- those are all in-sample, backward looking results. And none of the results have more than a handful of trades. Beware of making estimates of future performance based on in-sample results (even when there are a lot of data points, but that is another topic).

4. What are the results when a period of several years is analyzed, the best parameter values select based on those years, and the future results computed? Since the moving average lengths are so long -- typically around 24 months -- the in-sample period must be longer.

Five years is a long enough in-sample period to get some results. But, in part due to the edge effects of the walk forward process, one year is not a long enough out-of-sample period.

An in-sample length of six years, out-of-sample three years, starts to give results, but the out-of-sample results are not profitable.

Six years and four years gives only two steps -- one profitable, the other not.

My conclusion is that we cannot determine whether a moving average crossover system based on monthly bars is likely to be profitable in the future or not. Sixteen years of monthly data -- 192 data points is insufficient to allow meaningful validation.

What next?

1. Add more data. We might think that adding data from earlier periods will help. Testing to find out what moving average lengths worked best in the 1930s through the 1980s might be interesting, but it is still in-sample and has no value in estimating future performance. But we could run walk forward tests beginning at an earlier date and observe more steps. Whether those results look promising or not, we are still stuck with using the data we have from 1993 through the current. So we might find out what the best lengths of the in-sample and out-of-sample periods are. But we already know that the system is not sufficiently profitable or stable over the past eight years or more to actually trade it or to use it as a filter.

2. Use short bars. Moving to weekly bars increases the amount of data to about 830 data points. Moving to daily bars increases the amopunt of data to about 4000 data points.

Using weekly data, walking forward, two years in-sample, one year out-of-sample, long only, RRR as the objective function. The results are pretty promising. The system is profitable in nine OOS periods, unprofitable in three, and does not trade in two. But the values of the parameters are the interesting part. In 11 of the 15 OOS periods, the values are backwards. That is, be long when the faster moving average is below the slower moving average, and go flat when it crosses from below to above. The values of the parameters vary quite a lot, with four of the steps having both lengths greater than 60 weeks. In the steps where the moving average periods are both below 30, 10 of 11 have the values backwards -- often the faster period is one bar.

The conclusion I draw is that the traditional thoughts on using moving average crossovers as filters for trading systems have it backwards. When run as a walk forward test, which is the only way to estimate what future performance is likely to be, the S&P 500 is mean reverting, not trend following.

Thanks for listening,
Howard

2. ## Re: Testing a moving average crossover system

Hi Howard

I noticed you touched on moving average crossover systems in your book, and that they are backwards. Personally i have never come across a moving average crossover system that is profitable. This is a shame as most newcomers to system development and trading always seem to start with looking at MACD.. This may be why the systems are never profitable, because the amount of people using them is too much for the market and it can start to change.

I believe there is a good reason moving average crossover systems are backwards, and maybe a slight addition to your system could make it profitable. If you have a 116 and a 15 moving average, which you optimized for CAR (which i personally use as a yard stick), if the market is trending well enough, like it was from say 2003-2007, they 15day moving average will allow you to buy in pullbacks in part of a larger trend, as long as the price reverses and then rockets ahead of the 15day moving average, as seen in the image attached, a profit can be made.

*daily chart

*weekly chart

*monthly chart

There are some major simaliarites in all three of those charts, each one of them the price dips, and then rockets out of that dip before the moving average can catch up. Allowing us to swing trade - rather than following a trend.

I really don't believe that moving average systems will ever be profitable, simply because of the number of people who revert to it at the start of trading. The only sort of system that uses a moving average crossover will be a backwards one, because that is the type of system that over the years, will be abused the least.

I truly believe that if they are to be profitable, the focus needs to be shifted from trend following, buying when a fast ma crosses a slow ma, to being backwards and swing trading on fast price movements. The system needs to find minor pullbacks that will give the price action enough room to move up at a fast rate, so that once the fast MA catches up the price is already well ahead of both averages.

Cheers

3. ## Re: Testing a moving average crossover system

Hi Howard,

It is interesting the conclusions that you came to regarding the S&P500 being mean reverting rather than trend following. I developed my systems using ASX data many years ago and always found that trend following provided excellent results. Indeed, there are quite a few traders here in this forum and others who have done very well out of trend following systems (some even using weekly data). I had always thought that all markets were the same until I read your book (QTS) where you mentioned that the S&P 500 tended to mean revert and that the standard MA crossover methodology was in fact the incorrect way to apply it.

I got access to some US data and tested my systems on them and sure enough, they totally bombed out. They all went from >30% CAR from the ASX to <-30% CAR when used on the S&P500 stocks. Never in my wildest dreams did I anticipate that the difference between the ASX stocks and the S&P 500 stocks to be so great. The S&P500 stocks simply do not want to move !!!

I then spent quite a bit of time developing a mean reversion system, specifically to trade the S&P500 stocks. I now currently have a MR system that I am happy with and I thought that the acid test would be how it would perform using ASX data. Sure enough, the results came back almost identical to my previous experiment, the MR system totally bombed out using ASX data, with a CAR of -20%.

I used to think that one should be able to develop a sufficiently robust system to trade across all markets, whether they be futures, commdities, stocks etc etc, after all a price and volume chart is a price and volume chart right ?? Well from what I have seen, the answer is no. Each market has its own distinct characteristics and there are right and wrong ways to trade each market.

So in conclusion, I agree with your findings re S&P500 stocks. I would however be very interested to see what conclusions other traders would come to when they apply their trend following systems on S&P500 data.

4. ## Re: Testing a moving average crossover system

For some reason I am getting Error 30 Syntax with

OptimizerSetEngine("cmae");

I have looked at this for quite a while and do not see the error.

Thanks

5. ## Re: Testing a moving average crossover system

I have a book by Hill, Pruitt and Hill, called 'The Ultimate Trading Guide'.

The research done by the authors indicates that the Moving Average crossover system may be the second most profitable over a long period, with a Donchian Channel system being the most profitable.

There may well be great merit in developing this application and refining it.

6. ## Re: Testing a moving average crossover system

Originally Posted by daveM
For some reason I am getting Error 30 Syntax with

OptimizerSetEngine("cmae");

I have looked at this for quite a while and do not see the error.

Thanks
Hi Dave --

It may be that you are using an old version of AmiBroker. Bring it up to at least 5.20 and that statement will work fine.

Thanks,
Howard

7. ## Re: Testing a moving average crossover system

Originally Posted by daveM
I have a book by Hill, Pruitt and Hill, called 'The Ultimate Trading Guide'.

The research done by the authors indicates that the Moving Average crossover system may be the second most profitable over a long period, with a Donchian Channel system being the most profitable.

There may well be great merit in developing this application and refining it.
Hi Dave --

I also have that book. And I have exchanged emails about it with John Hill.

1. Most of the analysis of systems in that book are on in-sample data. None is clearly identified as out-of-sample data. In-sample results are not valuable in predicting future performance. The creation of multi-systems is all done with in-sample data and is very misleading.

2. Both moving average crossover and Donchian breakout systems worked well from early days of trading commodities into the 1980s, then they began to break down. If you have a chance, run the Donchian breakout system on commodity data from the first contract to the present, and you will see the equity curve turn from rising to flat as the system was revealed, became easier to program, and became cheaper to trade.

Both moving average crossover and breakout are trend following systems. They depend on relatively large winning trades, but there are many more losing trades than winning trades.

The keys to successfully trading breakout systems are:
Be able to limit losses of losing trades.
Trade a portfolio of many commodities.
Have a lot of money in your trading account -- \$1,000,000 plus.
Systematically follow precise and aggressive position sizing methods.

Trend following depends on the market being traded trending, which happens more in less mature, less sophisticated markets than in more mature markets (US common stocks and stock indexes do not trend well at all). It can work well in well defined sectors, such as sector ETFs.

As always, do your own testing following good modeling and simulation techniques, including walk forward testing and statistical analysis of the out-of-sample results.

Thanks for listening,
Howard

8. ## Re: Testing a moving average crossover system

Howard.
Longterm long only trading methods are pretty popular with those with larger funds looking to self manage their investments.They dont take a lot of maintenance.

Have you had any experience with or ever looked at System switches which turn a system on or off in periods of less than idea trading conditions.In the example above prolonged Bearish periods?

9. ## Re: Testing a moving average crossover system

As no-one else has mentioned it, it must just be me, but when I read the following early on in Howards post......

When tested on the 500 stocks currently in the S&P 500, using end-of-day data from Quotes Plus that goes back to about 1993 (for companies with history that long)
it meant a totally useless test to me.

The survivorship bias built in makes this type of study meaningless to me.

Surely you would want to start with the stocks that constituted the S&P500 in 1993 and walk forward to have any meaning.

brty

10. ## Re: Testing a moving average crossover system

Originally Posted by brty
As no-one else has mentioned it, it must just be me, but when I read the following early on in Howards post......

it meant a totally useless test to me.

The survivorship bias built in makes this type of study meaningless to me.

Surely you would want to start with the stocks that constituted the S&P500 in 1993 and walk forward to have any meaning.

brty
Actually no.

If the moving average crossover system can't profit over possibly the most stable stocks from 1993-2009 (the survivors if you will), then there is no way that it can be a good system.

You don't have to remove survivorship bias to prove that a system will not work, you only have to remove it to prove that it will work.

11. ## Re: Testing a moving average crossover system

Originally Posted by bingk6
I used to think that one should be able to develop a sufficiently robust system to trade across all markets, whether they be futures, commdities, stocks etc etc, after all a price and volume chart is a price and volume chart right ?? Well from what I have seen, the answer is no. Each market has its own distinct characteristics and there are right and wrong ways to trade each market.
This has been my conclusion to, after looking at different Mrkts......

Chorlton

12. ## Re: Testing a moving average crossover system

You don't have to remove survivorship bias to prove that a system will not work, you only have to remove it to prove that it will work.
You may have misinterpreted my post, my point being that it is a meaningless test to do in the first place.

What is the point of doing any test if the results cannot be duplicated in the real world, all it does is waste time and brainpower. Far better/only place to start at, is the real world situation, as it existed at the time, and walk forward.

brty

13. ## Re: Testing a moving average crossover system

Originally Posted by bingk6
Hi Howard,

I used to think that one should be able to develop a sufficiently robust system to trade across all markets, whether they be futures, commdities, stocks etc etc, after all a price and volume chart is a price and volume chart right ?? Well from what I have seen, the answer is no. Each market has its own distinct characteristics and there are right and wrong ways to trade each market.

So in conclusion, I agree with your findings re S&P500 stocks. I would however be very interested to see what conclusions other traders would come to when they apply their trend following systems on S&P500 data.
How much is this difference an artifact
of the lens that reality is viewed with

maybe 99.999% of the things people test and back test
are useless

But that does not mean there are things , significant things that are not important.
It just probably means that the universe of the unimportant that people wallow in is so huge..

What do markets do ?

they consolidate
they trend
they consolidate
they trend
consolidate = congest = reversion

[B]after all a price and volume chart is a price and volume chart right ??
No PRICE IS PRICE and VOLUME IS VOLUME

BUT a chart is just a lens
which often just distorts
and makes differences seem to be where there are none
Great fodder for a back testing wilderness

motorway

14. ## Re: Testing a moving average crossover system

Originally Posted by Chorlton
This has been my conclusion to, after looking at different Mrkts......

Chorlton
Have heard this before but not found it myself. I suspect it could be another truism.

Originally Posted by brty

You may have misinterpreted my post, my point being that it is a meaningless test to do in the first place.

brty
Not if all the tickers are included.Trading or defunct.I don't clear any of my data for this reason.Its only 4 yrs old but getting better everyday.
If its not then I see your point.

Originally Posted by motorway
How much is this difference an artifact
of the lens that reality is viewed with

maybe 99.999% of the things people test and back test
are useless

But that does not mean there are things , significant things that are not important.
It just probably means that the universe of the unimportant that people wallow in is so huge..

What do markets do ?

they consolidate
they trend
they consolidate
they trend
consolidate = congest = reversion

No PRICE IS PRICE and VOLUME IS VOLUME

BUT a chart is just a lens
which often just distorts
and makes differences seem to be where there are none
Great fodder for a back testing wilderness

motorway
All true Motorway
but the point of developing a system is a far cry from trading in a discretionary manner.

Systems are simply an application of a set of conditions repeatedly over a given data set and a given time frame,with the goal of achieving a positive expectancy when traded R/Time.

15. ## Re: Testing a moving average crossover system

Originally Posted by tech/a
Have heard this before but not found it myself. I suspect it could be another truism.
Fair Point.

My comment was based on my own initial testing, between the Australian and UK Mrkts.

I looked at similar (longer) timeframes, entry and exit conditions and MM / Position Sizing models and found that a particular idea would work well in one Mrkt but poorly in the other.

16. ## Re: Testing a moving average crossover system

Howard

I have your book........ lots of good material therein..... especially for a person attempting to master Amibroker...

Being new to system writing, I want to use as few variables as possible and then gradually add some items as I gain some experience with the tools....

There is a lot to absorb...

I will like to post some results once I get to the stage where I feel I can add to the discussion.

Thanks again

dave

17. ## Re: Testing a moving average crossover system

A concern that I have with any system is that initial results may show some excellent profit or devastating loss........ based on a few trades that are a small portion of the whole.....

Would it be prudent to limit the size of the profits and losses from the outset so that the trade sizes would be uniform to begin with..... ie....evaluate to see if there is a good distribution of profitable trades throughout the test period....... and then look for increased profits one the basis of the system is established.

I am not of a math background so I am not sure if it makes sense........... lol

Someone else may have some insights.

quote from 'Mechanical Trading Systems'by Weissman

In his book, Design, Testing and Optimization of Trading Systems,
Robert Pardo discusses a phenomenon that he calls outlier curve fitting.
Outlier curve fitting occurs when a single trade makes up a disproportionate
percentage of a trading system’s profits (Pardo specifically warns
against performance histories in which a single trade accounts for over 30
percent of a system’s profits.9) Certain types of trading systems are more
susceptible to this problem than others. In general, because mean reversion
trading systems exit with profits when the market reverts to its mean, outlier
curve fitting is unlikely. By contrast, trend-following systems have a
much greater tendency to contain single, disproportionately large profitable

18. ## Re: Testing a moving average crossover system

Originally Posted by brty
As no-one else has mentioned it, it must just be me, but when I read the following early on in Howards post......

it meant a totally useless test to me.

The survivorship bias built in makes this type of study meaningless to me.

Surely you would want to start with the stocks that constituted the S&P500 in 1993 and walk forward to have any meaning.

brty
Brty --

You are correct. There is a significant survivorship bias from using the current constituents. If the members of the index were chosen at the beginning of each year and tested forward for a year, the results would be much poorer.

Thanks,
Howard

19. ## Re: Testing a moving average crossover system

Originally Posted by tech/a
Howard.
Longterm long only trading methods are pretty popular with those with larger funds looking to self manage their investments.They dont take a lot of maintenance.

Have you had any experience with or ever looked at System switches which turn a system on or off in periods of less than idea trading conditions.In the example above prolonged Bearish periods?
Hi Tech/a --

One of the reasons that long-only trend following systems have a good record over recent history is that the market has been in a once in a multi-century bull market for most of recent history. In my opinion, that is very unlikely to reoccur in any of our lifetimes.

We should be looking for / developing systems that can recognize conditions under which they do poorly and either adjust or go flat. And systems that trade short as well as long.

Thanks,
Howard

20. ## Re: Testing a moving average crossover system

Originally Posted by brty
As no-one else has mentioned it, it must just be me, but when I read the following early on in Howards post......

it meant a totally useless test to me.

The survivorship bias built in makes this type of study meaningless to me.

Surely you would want to start with the stocks that constituted the S&P500 in 1993 and walk forward to have any meaning.

brty
Hi Brty --

One more comment --

The test I performed did walk forward. The selection of the issues to test was those that presently make up the S&P 500 -- it could have been any group. The walk forward testing removes the survivorship bias of the issue selection. If it did not, your comments would apply to any list of issues being tested. Norgate Premium Data distributes data on issues that have been delisted as well as those that are trading today. You can, if you wish, include a selection of delisted issues in your test universe. It will act as a stress test, and may be worth doing.

Thanks,
Howard

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Aussie Stock Forums