I thought I'd try learning how to effectively back test. After 9 months on and off of weekends writing my own software (okay, so total investment of time was probably close to 100-120 hours)... I found Amibroker, which did about 95% of what I wanted... Talk about frustrating.
I have been playing around with it all day as I have been sick with a flu (not sure if it's normal flu or swine flu, but my fiance thinks it's man-flu) and I have been very surprised by some of the results.
None of my results have consistently returned above a "buy and hold" strategy so far, but the biggest surprise is that some trading strategies that have returned 65% right on some shares, will return 35% right on others. (e.g. one iteration of a DMS returned about 11% annualised returns on BHP, but lost 5% on AMC over the same period. I'm guessing because BHP moves in long slow trends, but AMC was a lot more volatile)
And some strategies thatt are returning 35% right, are making more profit than other strategies that are 65% right, simply because the losses are 1% and the gains are 12%.
I'm right now "brute forcing" this. Trying out as many systems as I can over as many shares as I can. But I know this is not an thorough way of doing things.
Can anyone give some tips on how to design an effective and systematic back testing system?
Appreciate any feedback on either backtesting or trading systems.