New posts

Reply to thread

Message

Hi Skate,

I'm not much of a programmer, so I can't see any obvious logic error apart from using "close". I would suggest that you got such different results because of your stock universe and the fact that you did this comparison with a single run for each scenario.
Path dependence would be a significant factor. It's only a guess, but if you compare the trades, there will be different stocks picked at least some of the time to make up the difference in performance. It is unlikely to be just the price difference between today's close and tomorrows open. In other words, scenario 1 is just a "lucky" sequence.

I always say that I'm not qualified to give advice and then I do so anyway. Here is my (unqualified) opinion:

while it is quite interesting and even exciting to play around with backtests, I would suggest most of it is a waste of time. Consider this, masses of people use Metastock, Amibroker, Ninjatrader, Metatrader, etc. The sort of system you are testing has probably been tried thousands of times just this week, and it's only Tuesday.
It would be discouraging to hear this. Luckily it's only my opinion, not proof of anything.

What you will need for a system is a concept that has an edge. Otherwise you rely on a bull market or luck.
It would be good to have an entry signal with an edge. Maybe you can also find an exit with an edge. If there are a lot of trading opportunities this way, then you are really getting somewhere.

Secondly, it should be a simple concept. I'm not a believer in a lot of AND and AND NOT in my Buy or Sell statements, but that's just me. I avoid using parameters as much as possible and always examine how the range affects the various results like CAR, DD, exposure, #Trades, Winrate %, ProfitFactor, Sharpe, etc.
Just picking a "sweet spot" in a parameter is a big no-no. Of course there will be a different result for every parameter value, but there should be a range of profitable values.

Most indicators provide no edge. Don't trust me on this because technical analysts will disagree with me. Find out for yourself. An edge should be quantifiable. See if you can measure the edge of your Donchian Channel breakout.

Lastly, as you probably know, Howard Bandy has some good advice on in-sample and out-of-sample backtesting, some on posts here on ASF. It's hard to fault the logic. But there is an interesting consequence to consider. Unfortunately this post is getting too long to explain it. It would be almost good enough for its own thread.

The above is still only my opinion.

<blockquote data-quote="Habakkuk" data-source="post: 862017" data-attributes="member: 60923">Hi Skate,I&#039;m not much of a programmer, so I can&#039;t see any obvious logic error apart from using &quot;close&quot;. I would suggest that you got such different results because of your stock universe and the fact that you did this comparison with a single run for each scenario.Path dependence would be a significant factor. It&#039;s only a guess, but if you compare the trades, there will be different stocks picked at least some of the time to make up the difference in performance. It is unlikely to be just the price difference between today&#039;s close and tomorrows open. In other words, scenario 1 is just a &quot;lucky&quot; sequence.I always say that I&#039;m not qualified to give advice and then I do so anyway. Here is my (unqualified) opinion:while it is quite interesting and even exciting to play around with backtests, I would suggest most of it is a waste of time. Consider this, masses of people use Metastock, Amibroker, Ninjatrader, Metatrader, etc. The sort of system you are testing has probably been tried thousands of times just this week, and it&#039;s only Tuesday.It would be discouraging to hear this. Luckily it&#039;s only my opinion, not proof of anything.What you will need for a system is a concept that has an edge. Otherwise you rely on a bull market or luck.It would be good to have an entry signal with an edge. Maybe you can also find an exit with an edge. If there are a lot of trading opportunities this way, then you are really getting somewhere.Secondly, it should be a simple concept. I&#039;m not a believer in a lot of AND and AND NOT in my Buy or Sell statements, but that&#039;s just me. I avoid using parameters as much as possible and always examine how the range affects the various results like CAR, DD, exposure, #Trades, Winrate %, ProfitFactor, Sharpe, etc.Just picking a &quot;sweet spot&quot; in a parameter is a big no-no. Of course there will be a different result for every parameter value, but there should be a range of profitable values.Most indicators provide no edge. Don&#039;t trust me on this because technical analysts will disagree with me. Find out for yourself. An edge should be quantifiable. See if you can measure the edge of your Donchian Channel breakout.Lastly, as you probably know, Howard Bandy has some good advice on in-sample and out-of-sample backtesting, some on posts here on ASF. It&#039;s hard to fault the logic. But there is an interesting consequence to consider. Unfortunately this post is getting too long to explain it. It would be almost good enough for its own thread.The above is still only my opinion.</blockquote>

[QUOTE="Habakkuk, post: 862017, member: 60923"] Hi Skate, I'm not much of a programmer, so I can't see any obvious logic error apart from using "close". I would suggest that you got such different results because of your stock universe and the fact that you did this comparison with a single run for each scenario. Path dependence would be a significant factor. It's only a guess, but if you compare the trades, there will be different stocks picked at least some of the time to make up the difference in performance. It is unlikely to be just the price difference between today's close and tomorrows open. In other words, scenario 1 is just a "lucky" sequence. I always say that I'm not qualified to give advice and then I do so anyway. Here is my (unqualified) opinion: while it is quite interesting and even exciting to play around with backtests, I would suggest most of it is a waste of time. Consider this, masses of people use Metastock, Amibroker, Ninjatrader, Metatrader, etc. The sort of system you are testing has probably been tried thousands of times just this week, and it's only Tuesday. It would be discouraging to hear this. Luckily it's only my opinion, not proof of anything. What you will need for a system is a concept that has an edge. Otherwise you rely on a bull market or luck. It would be good to have an entry signal with an edge. Maybe you can also find an exit with an edge. If there are a lot of trading opportunities this way, then you are really getting somewhere. Secondly, it should be a simple concept. I'm not a believer in a lot of AND and AND NOT in my Buy or Sell statements, but that's just me. I avoid using parameters as much as possible and always examine how the range affects the various results like CAR, DD, exposure, #Trades, Winrate %, ProfitFactor, Sharpe, etc. Just picking a "sweet spot" in a parameter is a big no-no. Of course there will be a different result for every parameter value, but there should be a range of profitable values. Most indicators provide no edge. Don't trust me on this because technical analysts will disagree with me. Find out for yourself. An edge should be quantifiable. See if you can measure the edge of your Donchian Channel breakout. Lastly, as you probably know, Howard Bandy has some good advice on in-sample and out-of-sample backtesting, some on posts here on ASF. It's hard to fault the logic. But there is an interesting consequence to consider. Unfortunately this post is getting too long to explain it. It would be almost good enough for its own thread. The above is still only my opinion. [/QUOTE]

Verification