Backtesting

Hi Guys,

I've posted here a few times before, mainly in regards to value investing and a model I was developing myself. I've done some trial and error backtesting however I just want to gain some thoughts on if what I have done so far carries any weight or its not an accurate measure on whether my method will provide superior returns.

First i'll run you through my methodology in basic terms. I've set up a screener to provide companies with high ROE and low D/E. I then run the financials and come up with my intrinsic value. I further screen all these companies by only concentrating on those which are greater then 15% below their intrinsic value. I have a scoring system that i've devised to rank the stocks and it leaves me with approx 7-8 stocks.

From these, i chose to look at 5 stocks. Now my plan is to update my original screen every sunday and my spreadsheet picks up if any of the stocks have changed details, the ones that have changed details, i update their financials and go from there. So basically once a week I should pick up the new opportunities, which means reporting periods are my main times of picking up these opportunities.

Therefore the most recent i've looked at are the financial from August/September 2010 for my screen and the financials i've looked at etc.

So what i've done is said well hey if I bought these 5 stocks at the start of november because I was late doing my analysis, I still would have made x amount if I bought $5,000 worth of each stock.

It worked out I roughly would have made about a 13.7% return in 2 months while the ASX moved about 0.5%. If I look at the 3 months, that return becomes about 27.2% vs the asx of about the same, 0.5%.

Now I understand testing this once doesn't prove my methodology is correct in any way, but just want to know if this is kinda going the right way about it, and if so I can then look at how my process would have worked after previous reporting periods.

Thoughts? BTW the 5 stocks were LYL, FGE, MML, NCK and ACR.

In all honesty your testing will be long arduous and tedious.

You can only forward test a very small set of stocks which will not give you any meaningful statistical results over such a small set.

You would need 100s if not 1000s to show a statistical edge.

Your method may well be sound but you can only tell by real time trial and error.

One of the reasons I trade technically.

I was kind of thinking the same thing tech that backtesting it would be a pretty long task and take quite a bit of time. I mean the spreadsheets i've developed would help considerably, but it would still take quite a while.

So your suggestion is to possibly just do as i've suggested (i.e. update every sunday) and then see if it continues to make superior returns over the next 6-12 months for starters?

Thanks for your input, i'll probably continue to just see how it goes for the next 6 months or so then start trading on it actively. Also once I started actually trading, I don't plan on going and just buying the stocks as soon as my process identifies them, would also look at the charts/technicals to try and add a bit of timing to my entry.

Thanks for your input, i'll probably continue to just see how it goes for the next 6 months or so then start trading on it actively. Also once I started actually trading, I don't plan on going and just buying the stocks as soon as my process identifies them, would also look at the charts/technicals to try and add a bit of timing to my entry.

Click to expand...

If your going to combine the two (which is sound).
Id buy with a tight stop and you may cop a few premature stop outs but once running should be pretty easy to maintain. (Hence timing)
You could go ahead and trade without terminal damage to your trading account if things Dont pan out.

I trade short term discretionary and cannot test the discretionary components.
I trade it aggressively but I also know what's needed with a winning trading method.

(1) Small losses
(2) Big wins
(3) More wins than losses + (1) preferably
(4) A larger portfolio constantly monitored (10-15 trades).
(5) Constant pruning of Flat trades

This account is + 55% since I started trading it in October.

So If you can incorporate the above in your trading you should be able to trade and forward test with some sort of risk mitigation and profit opportunity.

Interested in how you go.

I think it would be accetable to rely on back test only. Forward testing using fundamental information sounds a bit funny. But you will need to back test over longer periods (e.g. during market downturn).

What you have here may be a valid entry strategy. But you need to think about exit strategies and risk management as well.

For exit - may be hitting your intrinsic value then place a trailing stop? You will also need an exit strategy which says something like 'no bad news that invalidate the analysis assumptions'.

For risk management - sensible position sizing (e.g. <15% of account in any one stock) would be a good minimum. A stop loss (e.g. <5% loss in capital) 'may' be helpful.

Also, when you back test you might have to manually monitor the above exit/risk management conditions which makes it an even more difficult process.

Kermit, what about a slightly different approach: loosen your fundamental parameters to allow say 50-100 stocks per week as 'acceptable'. Then trade these with a technicaL approach. Just use a simple parabolic stop and reverse on the Ords as your entry/exit. Should be very easy to test. Tell us what results you get. I'd do it myself but don't have Lincoln or anything similar.

Very thoughtful post, and I am getting allot from it. The one thing that stands out about your method, which is probably a sound approch, but your a person who is afraid to 'Sell'. You compensate this feeling with in depth research to reason why you should 'Buy' in the first place. Never be afraid to sell, you need to feel comfortable about selling, then it automatically becomes easy to 'Buy' an opportunity when it presents itself. Think about it youself what I'm saying, if you over analyse your 'Buying' then there's no way your going to be able to sell. You'll just freeze at your screen. Trading should be effortless.

To skc and Market Depth (as your both on a similar kind of path)

You both very much on target, I haven't really explored the sell side of the transaction and/or formulated an actual trading plan in relation to the stocks. I think i'm pretty much set on my process of identifying the value stocks that I want to pursue further, but haven't explored the trading plan / exit strategy side of things.

I had a bit of a think on it over lunch and came up with the following as a bit of a starting point:

- Stop-loss of 5% once stock has been bought on technicals
- Trailing stop of 5% once stock either :
a) surpasses the intrinsic value or
b) has achieved a gain of more than 50%

Only having a trailing stop on intrinsic value as my valuation technique isn't exactly perfect (what is in life?) so sometimes the intrinsic value can be quite a long way from the current price, in which case i'd be more than content with making a 50% gain on my winners.

Thoughts?

To Gringotts, I'm not really that interested in loosening my parameters and having even more stock to gather the information on and plug into my spreadsheet. At the moment my initial screen identifies 73 stocks which is plenty for me to cover initially anyway.

To tech/a, glad to see your strategy is working well for you. Agree with the 5 essentials that you've highlighted and they sound simple but obviously getting them to work in practice is a lot harder. Hopefully as i progress with my process i can track my trades and see how successful it can be. I think I saw in another thread someone posted a track record of their trading over the last 10 or so years and it had a number of statistics etc. I think once I start trading i'll make some form of a spreadsheet like this to also track my record.

If I have time i'll find the thread/post so you can see what i'm on about.

Thanks for your comments guys keep em coming.

EDIT: It was your post tech/a in the "Anybody here successful?" thread. I've managed to turn what you posted into an automated excel spreadsheet that just needs the trades punched in and spits out all those stats.

I've had a quick look at the stocks you mention kermit. All of them are in 'Uptrend' which is good. So your screen is picking stocks going the right way. I'll list a few other points on each, just so you get an idea of how things maybe going for your stock picks, and what stands out for me on an 'Eyeball' type test. I didn't spend long looking maybe 1 minute per chart, and jotted down a few points.

MML-Uptrend, sideways over the last 2 months, possible breakout, Good liquidity.
Resistance $6.90 Support $6.20

LYL-Strong Uptrend, Approching a 3 year 'HIGH', Lower liquidity

FGE- Uptrend, Small Double top, Support $4.50 Resistance $5.00 Good liquidity

NCK- Uptrend, low liquidity

ACR- Uptrend, Good liquidity, trading sideways for the last month. Support $3.50 Resistance $3.65 possible breakout.

That was just a quick glance, no more than 5 minutes. No volume anaysis for this quick first round search. All charts present well, all in uptrend. For me NCK would be scratched due to low liquidity, but this would come down to position size.

Thanks Market Depth,

A point to note, is that these 5 stocks would have been flagged by my scanner and spreadsheet a week or two after their last round of financial reporting (August-September-October) so the uptrend you've noticed would have been captured already had they been bought into back then. Obviously hindsight is a great thing to have, and its good to see the 5 of them are still trending up and you can't really see any issues with them in your quick glance.

Agree with NCK being a possible scratching due to liquidity, something I probably need to be more aware of when looking at the charts once I start actually trading.

The big test will be once I start actively updating my screen and spreadsheet weekly to see if any new stocks move into my final list and how they move after that. It seems these 5 have done well so far and still have some potential.

Once again, thanks

kermit345 said:

To skc and Market Depth (as your both on a similar kind of path)

You both very much on target, I haven't really explored the sell side of the transaction and/or formulated an actual trading plan in relation to the stocks. I think i'm pretty much set on my process of identifying the value stocks that I want to pursue further, but haven't explored the trading plan / exit strategy side of things.

I had a bit of a think on it over lunch and came up with the following as a bit of a starting point:

- Stop-loss of 5% once stock has been bought on technicals
- Trailing stop of 5% once stock either :
a) surpasses the intrinsic value or
b) has achieved a gain of more than 50%

Only having a trailing stop on intrinsic value as my valuation technique isn't exactly perfect (what is in life?) so sometimes the intrinsic value can be quite a long way from the current price, in which case i'd be more than content with making a 50% gain on my winners.

Thoughts?

Click to expand...

Entry and exit strategies need to be coherent... you make an entry for a particular reason, and you should exit when that reason is no longer valid. In addition, you adopt sensible position sizing just in case you are wrong with your analysis or some blackswan event happens.

A 5% stop loss seems totally random, while 50% gain may be too small if a stock wants to run.

Ask yourself why are you doing a fundamental stock scan? Is it so you could capture the difference between the SP vs IV? If so your exit should be based on what this difference is. If you want to adopt random stop loss and profit targets, then simply cut the fundamental screening part and just do some technical analysis.

Market Depth said:

Very thoughtful post, and I am getting allot from it. The one thing that stands out about your method, which is probably a sound approch, but your a person who is afraid to 'Sell'. You compensate this feeling with in depth research to reason why you should 'Buy' in the first place. Never be afraid to sell, you need to feel comfortable about selling, then it automatically becomes easy to 'Buy' an opportunity when it presents itself. Think about it youself what I'm saying, if you over analyse your 'Buying' then there's no way your going to be able to sell. You'll just freeze at your screen. Trading should be effortless.

Click to expand...

When you say "you", do you mean the OP or people in general? Where did get the idea that increased research leads to difficulty in selling?

Thanks skc, that makes sense, will take that into consideration.

I think Market Depth was eluding to the fact that the more you research a company and come to know the ins and outs of it, a lot of people will gain an emotional attachment and start to put the rosey coloured glasses on where everything they read about the company looks positive and it becomes hard to detach.

Obviously if your disciplined then this wouldn't be the case and if you form no emotional or mental attachment to a company then its easy to break away from them. Although i'd say for a lot of people it is easier said then done, myself included. (Probably why if i outline and adopt an actual trading strategy within my process, it could help me further).

skc said:

When you say "you", do you mean the OP or people in general? Where did get the idea that increased research leads to difficulty in selling?

Click to expand...

The reference was to kermit. Although I've seen this type of senario hundreds of times in other traders. I've even had the terrible task of telling these traders that their services are no longer required. Or at the very least, their trading allowance will be dramatically cut back and strict supervision applied to their trading. Margin Calls, locking trading accounts, stockbroker applied forced 'Sells'. Done all that. Ducked my fair share of right hooks. Even been on the receiving end of a few that made contact.

Nothing wrong with quality sound research. Spotting something within a company that no one else has come to elvaluate is the name of the game. What kermit has to do first is
1. Buy something, based on his research
2. sell that something in the future at or above his estimate. Based on his research

If in the meatime the stock price moves lower, kermit is faced with some more decisions
1a. trust his analysis that the stock is indeed worth what he researched, and hold.
2b. Average down his postion, based on his answer to the first question.
3. Sell and cut his loss. Watch the stock tick down and buy back in once it has reversed. Again keeping his research target price in mind.

So answers
1. 'Buy' something, based on research that's pretty easy, everyone can be a buyer.
2. 'Sell' something at or above target. Seems straight forward, but a little harder, as only holders of that stock can be sellers. And Will the market 'See' the value that kermit sees?

1a The answer doesn't involve any selling
2b Again it doesn't involve selling
3 This does involve a 'Sell', the most difficult sell of all.

Selling can be difficult for some people, even when based on quality sound research. The market might not be as 'Intune' with the potential valuation, that kermit sees. The market 'might' see the value and I hope it does. And the price moves in his favour. Selling is harder for some, but easier for others, experience makes it easier.