Thanks for the replies guys.
Skc, the system is essentially a trend follower but looks for a specific type of trend behaviour. The code’s made up of 4 filters (volume and index based) and 4 similar variables (again, fairly straight-forward code).
I’m aware of ways to get around the problem, I’m just after people’s opinions on how important you think it is during system design. A lot of the more technical stuff I’ve read on systems design implies it’s an absolute necessity for all types of systems design, but this doesn’t fit with what I’d expect.
I can understand it if your trying to trade a 20% win rate, 14 avg R/R system with a trade log of 200 trades – obviously small changes in the code that result in the trades changing could have massive effects on the outcome. In addition, the more restrictive/exhaustive the code (and the lower the trade frequency), the easier it is to find a single bit of code that will fit the data set very well.
But again, I think the relevance of OOS testing would be directly related to the number of trades in the log, the type of system you’re trading, the win rate and RR, and the smoothness of your equity curve – it doesn’t seem as clear cut as “you must always use OOS testing regardless of the situation”.
Tech, the system’s been Monte Carlo tested. In live testing it’s been trading well within the bounds expected. The main problem I’m having is that it generally buys as a counter-intuitive (read: scary) point in a trend – and this means you need to have a lot of confidence in it. I’m going to do some more testing over the coming weeks on different markets and might play about with the parameters to see the effect.
Hey GB, I realize I don’t have to wait six months but because I’ve tweeked the thing based on current data, I’d need six(?) months of fresh data to consider it OOS testing.
Thanks for the input guys. I’ll let you know how I go.